Robust radar-based gesture-recognition by user equipment

ABSTRACT

Systems and techniques are described for robust radar-based gesture-recognition. A radar system detects radar-based gestures on behalf of application subscribers. A state machine transitions between multiple states based on inertial sensor data. A no-gating state enables the radar system to output radar-based gestures to application subscribers. The state machine also includes a soft-gating state that prevents the radar system from outputting the radar-based gestures to the application subscribers. A hard-gating state prevents the radar system from detecting radar-based gestures altogether. The techniques and systems enable the radar system to determine when not to perform gesture-recognition, enabling user equipment to automatically reconfigure the radar system to meet user demand. By so doing, the techniques conserve power, improve accuracy, or reduce latency relative to many common techniques and systems for radar-based gesture-recognition.

RELATED APPLICATIONS

This application is a continuation of and claims priority to U.S.application Ser. No. 16/886,626, filed on May 28, 2020, which in turn isa continuation in-part of and claims priority to International PatentApplication Serial Nos. PCT/US2019/055731, filed on Oct. 10, 2019;PCT/US2019/053676, filed on Sep. 27, 2019; PCT/US2019/049208, filed onAug. 30, 2019; PCT/US2019/049212, filed on Aug. 30, 2019; andPCT/US2019/049216, filed on Aug. 30, 2019; which also claims priority toU.S. Provisional Application Ser. No. 62/894,566, filed on Aug. 30,2019, and claims priority to U.S. Provisional Application Ser. No.62/879,361, filed on Jul. 26, 2019, the disclosures of which areincorporated herein by reference in their entireties.

BACKGROUND

Some computing devices (also referred to as “user equipment”) include aradar system for detecting input. For example, the radar system providesa radar field from which the radar system recognizes two-dimensional andthree-dimensional (also referred to as “touch-independent”) radar-basedgestures made within or through the radar field. The radar system mayconstantly evaluate reflections within the radar field, frequentlytransitioning into a gesture-recognition state to interpret what couldbe radar-based gesture inputs. Transitioning to a gesture-recognitionstate in response to an unintended or false-positive radar input,however, wastes electrical power and may cause a malfunction if amistakenly-recognized radar-based gesture triggers or is used to performa function.

SUMMARY

This document describes techniques and systems for radar-basedgesture-recognition with context-sensitive gating and othercontext-sensitive controls. The techniques and systems use sensor datafrom a plurality of sensors to define the context of a user equipment.The plurality of sensors may include low-power sensor devices such as aninertial measurement unit (IMU) and exclude high-power sensor devices,such as a camera. The sensor data can be inertial sensor data from anIMU, proximity data from a proximity sensor, radar data from a radarsystem, or any other sensor data. The sensor data defines the context ofthe user equipment, such as a user activity or characteristic of acomputing environment. In certain contexts, when a radar system isunreliable or less reliable for radar-based gesture-recognition, thetechniques and systems enable the user equipment to automaticallydisable or “gate” radar-based gesture-recognition. To do so, the userequipment may restrict inputs to, or outputs from, a gesture-recognitionmodel. The user equipment may also disable the gesture-recognition modelto prevent the radar system from performing radar-basedgesture-recognition altogether. The user equipment can re-enableradar-based gesture-recognition when the context changes to a differentcontext that is not likely to cause errors in gesture-recognitions orcause false positives. If the user equipment is operating in a contextwhere radar-based gestures are unlikely, the user equipmentautomatically gates gesture-recognitions. Gating thegesture-recognitions prevents applications or other subscribersexecuting at the user equipment from performing functions in response toradar inputs obtained while gating. By so doing, the techniques preventfalse-positive gesture-recognitions from triggering operations bysubscribers of the gesture-recognitions. Preventing false-positivesconserves power and improves usability and user satisfaction forcomputing systems using radar-based gesture-recognition systems.

For example, an apparatus is described including: a radar system thatdetects radar-based gestures on behalf of application subscribers; aninertial measurement unit that receives inertial sensor data; and astate machine that transitions between multiple states for controllingthe radar system based on the inertial sensor data and context-sensitivetransition functions, the state machine including: a no-gating state inwhich the state machine enables the radar system to output indicationsof the radar-based gestures to application subscribers; a soft-gatingstate in which the state machine prevents the radar system fromoutputting the indications of the radar-based gestures to theapplication subscribers; and a hard-gating state in which the statemachine prevents the radar system from detecting the radar-basedgestures.

This document also describes a method performed by the above-summarizedapparatus and other methods set forth herein as well ascomputer-readable media having instructions for performing the methodand other methods set forth herein. This document also describes systemsand means for performing these methods.

This summary is provided to introduce simplified concepts forradar-based gesture-recognition with context-sensitive gating and othercontext-sensitive controls, which is further described below in theDetailed Description and Drawings. This summary is not intended toidentify essential features of the claimed subject matter, nor is itintended for use in determining the scope of the claimed subject matter.

BRIEF DESCRIPTION OF THE DRAWINGS

The details of one or more aspects of radar-based gesture-recognitionwith context-sensitive gating and other context-sensitive controls aredescribed in this document with reference to the following drawings. Thesame numbers are used throughout the drawings to reference like featuresand components:

FIG. 1 illustrates an example environment in which techniques forradar-based gesture-recognition with context-sensitive gating and othercontext-sensitive controls can be implemented.

FIG. 2 illustrates an example of the authentication system set forth inFIG. 1 .

FIG. 3 illustrates an example user authenticated by the authenticationsystem of FIG. 2 .

FIG. 4 illustrates an implementation of the user equipment of FIG. 1that can alter states, including a power state of an authenticationsystem responsive to determinations of a user's intent to engage with auser equipment.

FIG. 5 illustrates example information, power, and access states of auser equipment.

FIG. 6-1 illustrates an example radar system as part of user equipment.

FIG. 6-2 illustrates an example transceiver and processor.

FIG. 6-3 illustrates an example relationship between power consumption,a gesture-frame update rate, and a response delay.

FIG. 6-4 illustrates an example framing structure.

FIG. 7 illustrates example arrangements of receiving antenna elementsfor the radar system of FIG. 6-1 .

FIG. 8 illustrates additional details of an example implementation ofthe radar system of FIG. 6-1 .

FIG. 9 illustrates an example scheme that can be implemented by theradar system of FIG. 6-1 .

FIG. 10 illustrates an example method for authentication managementthrough IMU and/or radar.

FIG. 11 illustrates an example scenario for authentication management.

FIG. 12 illustrates an example method for reducing a state of a userequipment.

FIG. 13 illustrates an example scenario for reducing a state of a userequipment.

FIG. 14 illustrates an example method for maintaining an authenticatedstate.

FIG. 15 illustrates an example scenario for maintaining an authenticatedstate.

FIG. 16 illustrates another example scenario for maintaining anauthenticated state.

FIG. 17 illustrates an example method for radar-basedgesture-recognition with context-sensitive gating and othercontext-sensitive controls.

FIG. 18 illustrates an example method for radar-basedgesture-recognition with context-sensitive gating and othercontext-sensitive controls.

FIG. 19 illustrates a decision tree that implements the methods of theFIGS. 17 and 18 .

FIG. 20 illustrates a state diagram for a state machine that implementsthe methods of the FIGS. 17 and 18 .

FIG. 21 illustrates a block diagram for implementing movement-basedgating of radar-based gesture-recognition.

DETAILED DESCRIPTION

Overview

This document describes techniques and systems for radar-basedgesture-recognition with context-sensitive gating and othercontext-sensitive controls. As an example, a user equipment (UE) (e.g.,a computing device) includes a radar system for, among other uses,detecting input from a user. The UE receives sensor data from aplurality of sensors, such as a proximity sensor or a movement sensor,to develop a context of the UE.

The context defines a user activity, device characteristics, or anoperating environment of the UE. The context can specify orientation,acceleration, position, or proximity to an object. Location,temperature, luminance, pressure, and other environmentalcharacteristics can also define a context. The plurality of sensors mayinclude a movement sensor, such as an inertial measurement unit (IMU)for generating inertial data defining movement of the UE. The pluralityof sensors can include a proximity sensor, a light sensor, or atemperature sensor, to name just a few. When the radar system isoperating in a proximity mode (with or without gesture-recognitionenabled), the radar system is the proximity sensor. The UE may relysensors that provide accurate sensor data while consuming as littleelectrical power as possible, especially for UE's relying on batterypower.

Based on the context defined by the sensor data, the UE determineswhether to prevent the radar system from recognizing radar-basedgestures and/or whether to prevent components of the UE from using arecognized radar-based gesture to perform a function. Gatinggesture-recognitions made by the radar system prevent the UE fromwasting computing resources and electrical power interpretingradar-based gestures or performing functions (even malfunctioning) inresponse to a gesture-recognition made from unintentional or non-userinput.

Without gating, a UE over-interprets radar-based gestures from radardata, thereby wasting computational resources processing false gesturesor even malfunctioning in response thereto. By gating the output fromthe radar system based on context, the disclosed techniques and systemsenable a UE to conserve power, improve accuracy, improve usersatisfaction and usability, or reduce latency relative to othertechniques and systems for radar-based gesture-recognitions.

By way of one example, assume that sensor data obtained by a smartphoneindicates a user is holding the smartphone. The techniques and systemsenable a radar system of the smartphone to recognize radar-basedgestures in this context as the likelihood is high that the user willinput radar-based gestures to the smartphone while holding it. Thesensor data subsequently indicates that the user is also walking withthe smartphone. The smartphone continues to recognize radar-basedgestures with the radar system in this context as well because evenwhile walking, the user is likely to want to intentionally gesture atthe smartphone while holding the smartphone. The sensor data nextindicates that the user is still walking but no longer holding thesmartphone, the smartphone is oriented away from the user, and/or thesmartphone is occluded by an object (e.g., a backpack compartment). Asthe user is not likely to interact with the smartphone while in thebackpack compartment, for example, the techniques and systems enable thesmartphone to disable the radar system, or at least tune the radarsystem to prevent the radar system from being used to recognizeradar-based gestures in this context. When the smartphone recognizes anew context, the smartphone reevaluates whether to enable radar-basedgesture-recognition and enables radar-based gesture-recognition by theradar system when the context is appropriate for radar-basedgesture-recognition.

Eventually, the user places the smartphone on a surface, such as a desk,and the sensor data indicates the user is not holding the smartphone andthe smartphone is oriented with the screen facing up. If proximity dataindicates the user is reaching over the smartphone, the smartphoneselectively enables or prevents the radar system from recognizingradar-based gestures based on what the user does next. If the smartphonedetects movement indicating the user is picking up the smartphone afterhaving reached over the smartphone, the smartphone recognizesradar-based gestures with the radar system. If the smartphone does notdetect movement indicating the user is picking up the smartphone afterhaving reached over the smartphone (e.g., the user is grabbing a cup ofcoffee on the desk next to the smartphone), the smartphone preventsgesture-recognition using the radar system.

These are only some examples of how the described techniques and devicesmay be used to gate radar-based gesture-recognitions. Other examples andimplementations are described throughout this document. The document nowturns to an example operating environment, after which example devices,methods, and systems are described.

Operating Environment

FIG. 1 illustrates an example environment 100 in which techniques forcontext-sensitive gating and other context-sensitive controls ofradar-based gesture-recognitions can be implemented. The exampleenvironment 100 includes a user equipment (UE) 102 (e.g., a smartphone),which includes, or is associated with, a radar system 104, a radarmanager 106, a plurality of sensors 108, a movement manager 110, a statemanager 112, an authentication system 114, and a display 116.

In the example environment 100, the radar system 104 provides a radarfield 118 by transmitting one or more radar signals or waveforms asdescribed below with reference to FIGS. 7-9 . The radar field 118 is avolume of space from which the radar system 104 can detect reflectionsof the radar signals and waveforms (e.g., radar signals and waveformsreflected from objects in the volume of space, also referred togenerally herein as radar data). The radar system 104 also enables theUE 102, or another electronic device, to sense and analyze this radardata from reflections within the radar field 118, for example, torecognize radar-based gestures (e.g., touch-independent gestures) madeby a user in the volume space. The radar field 118 may take any of avariety of shapes and forms. For example, a radar field 118 may have ashape as described with reference to FIGS. 1 and 7 . In other cases, theradar field 118 may take the shape of a radius extending from the radarsystem 104, a volume around the radar system 104 (e.g., a sphere, ahemisphere, a partial sphere, a beam, or a cone), or a non-uniform shape(e.g., to accommodate interference from obstructions in the radar field118). The radar field 118 may extend any of a variety of distances fromthe radar system 104 such as inches to twelve feet (less than a third ofa meter to four meters). The radar field 118 may be predefined,user-selectable, or determined via another method (e.g., based on powerrequirements, remaining battery life, or another factor).

The reflection from the user 120 in the radar field 118 enables theradar system 104 to determine various information about the user 120,such as the body position and posture of the user 120, which mayindicate a variety of different nonverbal body language cues, bodypositions, or body postures, which can be recognized by the radar system104 as touch-independent gestures made by the user 120. The cues,positions, and postures may include an absolute position or distance ofthe user 120 with reference to the UE 102, a change in the position ordistance of the user 120 with reference to the UE 102 (e.g., whether theuser 120 or the user's hand or object held by the user 120 is movingcloser to or farther from the UE 102), the velocity of the user 120(e.g., a hand or a non-user object) when moving toward or away from theUE 102, whether the user 120 turns toward or away from the UE 102,whether the user 120 leans toward, waves toward, reaches for, or pointsat the UE 102, and so forth. These reflections can also be analyzed todetermine, or to add confidence to, authentication, such as an identityof a human through analysis of the radar data (e.g., scattering centersof a user's face). These reflections can be used by the UE 102 to definea context (e.g., an operating environment of the UE 102) for performingcontext-sensitive gating and other context-sensitive controls ofradar-based gesture-recognitions. These reflections can also be used todetermine or to add confidence to, touch-independent gestures recognizedby the radar system 104 as the user 120 provides input to the UE 102.

The radar manager 106 is configured to determine, based on radar datafrom the radar system 104, a user's intent to engage, disengage, ormaintain engagement with the UE 102. A user's intent can be deduced fromtouch-independent gestures recognized by the radar system 104, e.g., thevarious cues, positions, postures, and distances/velocities noted above,such as based on an arm or hand gesture (e.g., a hand or arm reachtoward, swipe over), an eye gesture (e.g., a movement of eyes to lookat), or a head gesture (e.g., movement of a head or face oriented towardthe UE 102). For a hand or arms reach, the radar manager 106 determinesthat the user is reaching their hand or orienting their arm in such away as to indicate a likely intent to touch or pick up the UE 102.Examples include a user reaching toward a volume button on a wirelesslyattached speaker, a reach toward a wireless or wired mouse associatedwith a tablet computer, or a reach toward the UE 102 itself. This reachtoward can be determined based on a hand movement alone, an arm and handmovement, or an arm bending or straightening in a manner that permits ahand of the arm to touch or grab the UE 102.

A user's intent to engage can also be deduced based on a user's movementof their or their head or eyes to look at, or orient their or their facetoward, the UE 102 or, in some cases, an associated peripheral of the UE102. For a movement of a user's eyes to look toward the UE 102, theradar manager 106 determines that the user's eyes are looking in thedirection of the UE 102, such as through tracking of the user's eyes.For movement of the user's head to orient their or their face toward theUE 102 (e.g., a facial orientation), the radar manager 106 determinesthat various points (e.g., scattering centers as noted below) are noworiented such that the user's face is pointing toward the UE 102. Thus,a user need not perform an action designed to control or activate the UE102, such as activating (pressing) on a button on the UE 102, or atouch-dependent gesture (e.g., on a touchpad or screen) ortouch-independent gesture (e.g., using the radar system 104) in orderfor the radar manager 106 to determine that the user intends to engage(or disengage or maintain engagement) with the UE 102.

As noted above, the radar manager 106 is also configured to determine auser's intent to disengage with the UE 102. The radar manager 106determines a user's intent to disengage similarly to a user's intent toengage, though deduced from radar data indicating lack of atouch-independent gesture, or that the user's hand or arm is moving awayfrom the UE 102 (e.g., retracting), movement of eyes to look away from,or movement of the head or face away from the UE 102 (e.g., a facialorientation change away from looking at the UE 102). Additional mannersthrough which to determine a user's intent to disengage are not only theopposite or cessation of engagement noted above, but also radar dataindicating that the user has walked away, moved their or their body awayfrom, or has engaged with a different, unassociated object or device.Thus, the radar manager 106 may determine an intent to disengage withthe UE 102 based on determining an intent to engage, by the user, withsome other object, device, or user equipment. Assume, for example, thata user is looking at and interacting with a smartphone. Example intentsto engage that indicate an intent to disengage with that smartphoneinclude the user looking, instead of at the smartphone, at a televisionscreen, beginning to talk to a nearby physically-present person, orreaching toward another device with which engagement is likely toreplace the engagement with the smartphone, such as an e-book or mediaplayer.

The radar manager 106 is also configured to determine a user's intent tomaintain engagement with the UE 102. This maintaining of engagement canbe active or passive. For active engagement, the radar manager 106 maydetermine, based on radar data, that the user is interacting throughtouch-independent gestures, and so forth. The radar manager 106 may alsoor instead determine active engagement through non-radar data (e.g.,performed with assistance from other components of the UE 102). Thesenon-radar data include indications that the user is inputting data to orcontrolling the UE 102 or a peripheral. Thus, through touch, typing, oraudio data, the user is determined to be touching (e.g., tapping on asoft keyboard or performing a gesture) through a touch-screen input ofthe display 116, typing on a peripheral keyboard, or is determined to bedictating an audio input. For passive maintaining of engagement, theradar manager 106 determines, independently or through assistance ofother components of the UE 102, that the user is consuming content orproviding the UE 102 to others to consume content, such as pointingtheir or their face toward the UE 102, looking at the display 116, or isholding the UE 102 in such a way as to orient the UE 102's display to bevisible by the user or a third party. Other examples of maintainingpassive engagement include a user's presence, such as through the radarmanager 106 determining that the user 120 is within reach of (e.g., two,one and a half, one, or one-half of one meter from) the UE 102. Detailsof example ways in which the radar manager 106 determines a user'sintent to engage, disengage, or maintain engagement are described below.

Further still, the radar manager 106, using radar data from the radarsystem 104, may also determine gestures performed by a user. Thesegestures can involve the user touching some surface, such as a table,the display 116, or their or their shirt sleeve, or touch-independentgestures. Touch-independent gestures can be performed in the air, inthree dimensions, and/or without necessitating a hand or fingers touchan input device but are not precluded from touching some object. Thesegestures can be recognized or determined based on the radar dataobtained by the radar system 104 and then output to applications orother subscribers executing at the UE 102 or used as input to perform anoperation, such as to indicate engagement with, the UE 102.

Example gestures include those similar to sign language (e.g., ASL orAmerican Sign Language), which are varied, complex single hand ormulti-hand gestures, or simple multi-hand or single-hand gestures, suchas to swipe left, right, up, or down, flat-hand-raise or lower (e.g., toraise or lower music volume of the UE 102 or a television or stereo,controlled through the UE 102), or to swipe forward or backward (e.g.,left-to-right or right-to-left) to change music and video tracks, snoozealarms, dismiss phone calls, or even play games. These are but a few ofthe many example gestures and functions controllable by these gesturesand which are enabled through the radar system 104 and the radar manager106. Thus, while this document is directed to in some aspects toengagement and state management, nothing in this document should bemisconstrued to indicate that the engagement and state managementaspects cannot be used to additionally or alternatively configure theradar system 104 and the radar manager 106 to performgesture-recognition.

The display 116 can include any suitable display device, such as atouchscreen, a liquid crystal display (LCD), thin-film transistor (TFT)LCD, an in-plane switching (IPS) LCD, a capacitive touchscreen display,an organic light-emitting diode (OLED) display, an active-matrix organiclight-emitting diode (AMOLED) display, super AMOLED display, and soforth. As noted, the display 116 can be powered at various levels, suchas at full saturation with touch-input powered, reduced saturationwithout touch-input powered, and with low-saturation and low-power(e.g., a gray clock) or no power.

The plurality of sensors 108 can be any of a variety of sensor devicesconfigured to generate sensor data indicative of a context of the UE102, or in other words, an indication of an operating environment orsurroundings of the UE 102. The plurality of sensors 108 include aninertial measurement unit (IMU) to measure movement, which is heredefined to include specific force, angular rate, orientation,vibrations, acceleration, velocity, and position, including pitch, roll,and yaw for each of three axes (e.g., X, Y, and Z). An IMU is but oneexample of the sensors 108. Other examples of the plurality of sensors108 for sensing movement include an accelerometer, gyroscope, and/ormagnetometer. The plurality of sensors 108 can include a proximitysensor, a light sensor, a positioning sensor, a compass, a temperaturesensor, a barometric pressure sensor, or any other sensor to detectpresence or proximity to an object. The plurality of sensors 108 mayinclude a proximity sensor, such as the radar system 104, operating in aproximity mode as opposed to a gesture-recognition mode or another mode.

The UE 102 may rely primarily on battery power, and as such, theplurality of sensors 108 may exclude high-power sensors like cameras andinstead, primarily include low-power sensors that provide accuratesensor data for developing accurate contexts. By avoiding use ofhigh-power sensors like cameras to drive gating decisions, instead usingsensor data from low-power sensors like IMUS, the UE 102 operates moreefficiently, using less power to make gating decisions than if camerasare used.

The movement manager 110 is configured to determine, based on inertialdata or other sensor data obtained from the sensors 108, movements ofthe UE 102. The movement manager is configured to determine movements ofthe UE 102 to enable the UE 102 to define a context. Example movementsinclude the UE 102 being lifted (e.g., picked up), oriented toward oraway from the user 120, and vibrations. Example movements can indicatecessation of physical contact by the user 120 of the UE 102, placementof the UE 102 on a non-living object (e.g., a table, car console, coucharm, pillow, floor, docking station), and placement of the UE 102 withinan enclosed container, e.g., a pocket, bag, or purse. Further examplemovements include those indicating the UE 102 is being held, movementsindicating the UE 102 is being held by a person who is walking, riding abicycle, riding in a vehicle, or otherwise moving, movements indicatinghow the UE 102 is being held, such as a carry-orientation of being inlandscape, portrait-up, portrait-down, or a combination thereof. Examplemovements further include the UE 102 not being held, or movementsindicating the UE 102 is not being held but carried on the person who iswalking, etc.

These movements can indicate a user's potential engagement,disengagement, or maintained engagement with the UE 102. For example,the movement of the UE 102 may indicate that the user equipment ismoving or orienting toward or is being moved/oriented away from the user120, is moving too rapidly or changing movement too rapidly to beinteracted with for many likely types of user engagement, is being heldby the user 120 (via natural human movements, respiration, heartbeat),or is vibrating due to a mechanical or non-user source (e.g., avehicle's vibration, ambient sounds shaking the UE 102, music causingthe UE 102 to vibrate). Thus, orienting away, which would indicate apotential disengagement with the UE 102, may include an orientationchange of the UE 102 such that a prior orientation where the user 120was likely to have been looking at the display 116, is now unlikely tobe doing so. The user 120 typing or reading at one orientation, and thenturning the phone over, or sideways, or placing in a pocket, etc., isbut one example of a movement indicating an orienting away and thus apotential disengagement. Example movements that may indicate maintainedengagement include vibrations indicating that a user is maintaining ahold or placement of the UE 102 or is maintaining their or theirorientation relative to the UE 102 where that orientation previouslyindicated or was coincident with, engagement with the UE 102.

The radar system 104 relies on the radar manager 106, the movementmanager 110, and the sensors 108 to define a context of the UE 102 thatis used to drive gating decisions made by the radar system 104. Thesensor data generated by the sensors 108, in combination with themovements and the user intents determined by the movement manager 110and the radar manager 106, helps define the context of the UE 102.

Movements determined by the movement manager 110 may indicate how or ifa user 120 is interacting with the UE 102. Accelerations or vibrationsdetected by the movement manager 110 can correspond to similarvibrations and accelerations observed when the user 120 is walking orotherwise moving with the UE 102 and therefore the movements mayindicate how the user is walking or moving. Changes in movementdetermined by the movement manager 110 can indicate changes in carryingposition and orientation, as further information about how or if theuser 120 is interacting with the UE 102. Patterns of movement or lack ofmovement inferred by the movement manager 110 may be similar tomovements typically observed when the user 120 is viewing or holding theUE 102 in a holding context. The movements and patterns of movement canindicate a stowed context under such conditions when the UE 102 iscontained in a pocket of clothing worn by the user 120 or in a backpackor briefcase, an overhead storage bin in an airplane or train, a consoleor glove box of a vehicle, or other storage enclosure.

The context can be defined by other information beyond movement. Forexample, a proximity sensor or the radar system 104 can detect whetherthe radar system 104 (or other part of the UE 102) is occluded by anobject in proximity to the UE 102. Evidence of occlusion may indicatethe UE 102 is in a stowed context, and lack of occlusion may indicateotherwise. Other sensors such as ambient light sensors, barometers,location sensors, optical sensors, infrared sensors, and the like canprovide signals to the UE 102 that further define the operatingenvironment or context of the UE 102 to improve gesture-recognition andother described techniques. Relative elevations, shadows, ambientsounds, ambient temperatures, and the like are further examples ofsignals that can be captured by the radar system 104 through the sensors108 to enable the UE 102 to define a context.

The state manager 112 manages states of the UE 102, such as power,access, and information states, and in some examples, manages the statesbased on the context defined above. This management of the UE 102 andits components is performed based partly on determinations made by theradar manager 106 and the movement manager 110, sensor data from thesensors 108, and a context defined therefrom. For example, the statemanager 112 can manage power to a component of the authentication system114, such as by altering the UE 102's display 116 to power up inanticipation of receiving touch input from the user 120 to input apassword, a computer processor to perform calculations used inauthentication, or an imaging system to perform image-based facialauthentication, radar (e.g., the radar system 104), or other components.The state manager 112 may direct the radar manager 106 to place theradar system 104 in a proximity or a disabled mode when radar-basedgesture-recognition is gated and to place the radar system 104 in anenabled or a gesture-recognition mode when radar-basedgesture-recognition is not gated.

As noted, this managing of the UE 102 is based on determinations by theradar manager 106 and the movement manager 110, which determine anintent to engage, disengage, or maintain engagement and movement of theUE 102, respectively. The state manager 112 can do so based on thesedeterminations alone or also based on other information that defines thecontext of the UE 102, including a current state, current engagement,applications running and the content shown by these applications, and soforth. The state manager 112, by accounting for the context, can improvethe accuracy, robustness, and speed of an overall determination that theuser's intent is to engage, disengage, or maintain engagement with theUE 102.

The “multiple determinations” (e.g., that of the radar manager 106 andthe movement manager 110) to define a context can be performedconcurrently or in stages as part of managing the states of the UE 102,or one of these may alone be used. For example, assume that the UE 102is at a low-power state for components used to authenticate. The radarmanager 106 may determine that the user 120 is intending to authenticatewith the UE 102 based on a movement toward or a reach toward the UE 102.In some cases, this alone is considered by the state manager 112 to bean insufficient context for the state manager 112 to cause the UE 102 tobe altered to a high-power state (e.g., for authenticating, forinterpreting radar-based gestures). Thus, the state manager 112 cancause some of the authentication components to be powered up to anintermediate state, rather than a high-power state (e.g., the high-powerstate 504-1 of FIG. 5 ). For example, in cases where the authenticationsystem 114 uses infrared sensors to perform facial recognition, thestate manager 112 can power these sensors and the display 116 to ahigher power, in anticipation of authenticating the user, and in thecase of the display 116, indicating to the user that the UE 102 is“waking up” and therefore is increasingly responsive. As an additionalstep, the state manager 112 can wait until the movement manager 110determines that the context indicates the user has moved, picked up,lifted, and so forth the UE 102 before fully powering on theauthentication components, here the infrared sensors. While notrequired, the state manager 112 may cause the authentication to beattempted by the components without further input from the user, therebymaking authentication seamless for the user 120.

In some cases, however, the state manager 112 determines to power up orotherwise prepare the state of the UE 102 responsive to both inertialdata and radar data, e.g., the radar manager 106 determining that theuser intends to engage and the movement manager 110 determining that theuser is picking up the UE 102.

Thus, the state manager 112 can wait until a higher level of confidencethat the user's intent is to engage by picking up the UE 102, such as anindication by the movement manager 110 that the user has just started totouch the UE 102. In such a case, the state manager 112 may increasepower based on just the radar manager 116's determination but may do soto an intermediate-power level of a display or the authentication system114 or component thereof, instead of waiting until the movement manager110 indicates a touch by the user to fully power these components. Asnoted, however, the state manager 112 may alter states to higher powerlevels solely on determination of an intent to engage based on radardata or lower those levels solely on determination of an intent todisengage based on radar data.

One of many example ways in which the state manager 112 can managestates of the UE 102 is shown in FIG. 1 at example environments 100-1,100-2, and 100-3.

In the environment 100-1, assume that the user 120 is authenticated andthat the UE 102 is in a high-level state for power, access, andinformation. This authentication is indicated to the user 120 throughthe display 116 showing a high-saturation and high-luminosity starsymbol (shown in the environment 100-1 at 122). At the environment100-1, the user 120 places the UE 102 down on the table. This placing ofthe UE 102 on the table results in the sensors 108 sensing, and thenproviding inertial data, to the movement manager 110. The movementmanager 110 determines, based on this inertial data, that the UE 102 hasmoved but is now resting still. The UE 102 was in a moving context andis now in a stationary context. At this point the movement manager 110may pass this movement determination to the radar system 104, the radarmanager 106, or the state manager 112, but in either three cases this isa data point for determining whether or not to reduce the states fromhigh levels to intermediate or lower levels. As noted, reducing thesestates can save power, keep information private and access secure, andstill provide a seamless user experience for the user 120. For example,based on the movement determination, the radar system 104 forms acontext for managing its output. In the environment 100-1, where someslight movement is detected, but the user 120 is not holding the UE 102,the radar system 104 can determine the environment 100-1 where the UE120 is resting on a flat, stationary surface. Because the context forthe UE 120 is a stationary context, other than slight movement, thecontext satisfies requirements for radar-based gesture-recognition.

Continuing this example, consider environment 100-2, where the user 120retracts their hand from the UE 102. This retraction is sensed andanalyzed by the radar system 104 and the radar manager 106,respectively. By so doing, the radar manager 106 determines that theuser 120 is intending to disengage from the UE 102. Based on thisdetermination from the radar manager 106 and the movement determinationfrom the movement manager 110, the state manager 112 may reduce one ormore of the states of the UE 102. Here this reduction is intended tocorrespond to the user's 120 level of engagement with the UE 102. Thisreduction by the state manager 112 is to an intermediate level of powerby reducing the saturation and luminosity of the display 116, shown at alow-luminosity and saturation star symbol (shown at 124). Note that thestate manager 112 can reduce the states to a low level of power, access,and/or information, but here the state manager 112 reduces the states toan intermediate level, as the intent to disengage from the radar manager106 indicates that the user 120 is retracting their or their arm, butthat their body is still oriented toward the UE 102 and the user 120 isstill looking at the UE 102. This is one example of tailoring the statesto a user's engagement, as the retraction indicates a certain level ofdisengagement, but may, on its own, indicate either some continuingengagement or some level of uncertainty in the disengagementdetermination by the radar manager 106. For example, the retractiondetermination can be used as proximity information for defining a secondcontext of the UE 102. The radar system 104 forms the second context formanaging its gesture-recognition capability in the environment 100-2.Where a retract is detected without any movement to the UE 102, theradar system 104 can determine the environment 100-2 is a stowed contextthat does not satisfy requirements for touch-independentgesture-recognition as the context indicates the device is now out ofthe user's 120 reach.

Concluding this example, consider the environment 100-3. Here the user120 is reading their book, with the UE 102 lying on the table. The user120 orients their body at some angle away from the UE 102 and towardtheir book, and the user 120 is looking at the book, not the UE 102.Based on this additional information about the user's 120 orientation,the radar manager 106 determines that the user 120 is intending to (andlikely has) disengaged from the UE 102. At this point, the radar manager106 provides this additional intent to disengage determination to thestate manager 112, which then lowers the states of the UE 102 to a lowlevel, shown with lower power usage at the display 116 (showing onlytime of day at low-luminosity and saturation at 126). While not shown,the state manager 112 also de-authenticates the user 120 (e.g., locksthe UE 102). This additional information about the user's 120orientation or the determination that the user 120 is intending todisengage from the UE 102 can be used as proximity information fordefining a third context of the UE 102. The radar system 104 forms thethird context in response to detecting an intent to disengage and read abook. The radar system 104 can determine the environment 100-3 is a notcontext that satisfies requirements for radar-based gesture-recognitionas the context indicates the device is now out of the user's 120proximate influence.

As shown in this example, the techniques described herein can managestates of a user equipment to provide a seamless user experienceinvolving authentication and radar-based gesture-recognition. Thetechniques enable doing so with reduced power consumption and greaterprivacy and security over other techniques. The state management canachieve maintaining or increasing levels of power, access, andinformation. As further shown, without gating, the UE 102 mayover-interpret radar-based gestures from radar data obtained by theradar system 104, thereby wasting computational resources processing andsubsequently discarding false gestures. By gating thegesture-recognitions of the radar system 104 based on context, thedisclosed techniques and systems enable the UE 102 to conserve power,improve accuracy, or reduce latency interpreting and responding toradar-based inputs, relative to other techniques and systems forradar-based gesture-recognitions.

In more detail, consider one example of the authentication system 114,shown in FIG. 2 . This is but one example, as other authenticationsystems controllable by the state manager 112 are considered, such aspassword-entry through a touch-sensitive display, radar authenticationusing the radar system 104, or a finger-print reader, to name just afew.

This example of the authentication system 114 is illustrated showing aninterior 200 of the UE 102 (shown as a smartphone). In the depictedconfiguration, the UE 102 includes a radar integrated circuit 202 of theradar system 104, a speaker 204, a front-facing camera 206, and asexamples of the sensors 108, a proximity sensor 208 and an ambient lightsensor 210. As further examples of the sensors 108, the UE 102 alsoincludes a face-unlock sensor 212, which includes a near-infrared (NIR)flood illuminator 214 and a near-infrared (NIR) dot projector 216, bothof which project infrared or near-infrared light on a user. Theface-unlock sensor 212 also includes two NIR cameras 218-1 and 218-2,which are positioned on opposite sides of the UE 102. The NIR cameras218-1 and 218-2 sense the infrared and near-infrared light that isreflected by the user. This reflected near-infrared light can be used todetermine facial features and, with these features, determine if theuser is authentic based on comparison with previously-storedfacial-feature information. The NIR flood illuminator 214, for example,“floods” an environment with NIR light, which provides, on receiving thereflection from the user (and other objects), an image. This imageincludes, even in low or no ambient light, the face of a user, and thuscan be used to determine facial features. The NIR dot projector 216provides NIR light reflections that can be analyzed to determine depthof objects, including features of a user's face. Thus, a depth map(e.g., a spectrum depth map) for the user can be created (e.g.,previously when setting up facial authentication) and a current depthmap can be determined and compared to the stored, previously-createddepth map. This depth map aids in preventing authentication of a pictureor other two-dimensional rendering of a user's face (rather than theperson's actual face).

This mapping of a user's facial features can be stored securely on theUE 102 and, based on a user's preferences, be both secure on the UE 102and prevented from being made available to external entities.

The authentication system 114 includes the face-unlock sensor 212, butcan also include other components, such as the front-facing camera 206,the proximity sensor 208 and the ambient light sensor 210, as well asprocessors to analyze the data, memory (which may have multiple powerstates as well) to store, cache, or buffer the sensor data, and soforth.

The face-unlock sensor 212 senses IR (infrared) and NIR (near-infrared)data to perform facial recognition, which is one way in which thetechniques may authenticate the user and therefore alter an access state(e.g., to unlock the UE 102) as noted in the methods described below. Toconserve power, the face-unlock sensor 212 operates in a low-power state(which can also be simply off) when not in use. In particular, the NIRflood illuminator 214 and the NIR dot projector 216 do not radiate inthe off-state. However, a warm-up sequence associated with transitioningfrom a low or no-power state to an intermediate-power state and/or ahigh-power state can be used for the NIR flood illuminator 214 and theNIR dot projector 216. By powering up one or both of these components,the latency in authenticating the user can be reduced, sometimes by ahalf-second or more. Given the tens or even hundreds of times many usersauthenticate their devices each day, this can save the users time andimprove their experience. As noted herein, this time delay is reduced bythe radar manager 106 determining that the user is intending to engagewith their or their device based on radar data provided by the radarsystem 104. This is managed by the state manager 112. In effect, thetechniques proactively detect the user's intent to engage and initiatethe warm-up sequence. The techniques may do so even prior to the usertouching the UE 102, though this is not required. Thus, the techniquesenable the NIR flood illuminator 214 and the NIR dot projector 216 to besufficiently powered to be used in authenticating the user, whichreduces time spent by the user waiting for facial recognition tocomplete.

Before moving on to other components in the UE 102, consider an aspectof the face-unlock sensor 212. This example component of theauthentication system 114 can authenticate a user using facialrecognition in as little as ten degrees relative to the plane of thedisplay 116. Thus, the user need not pick up the phone and turn thesensors to their or their face, such as at an angle of 70 to 110 or 80to 100 degrees, instead, the authentication system 114, using theface-unlock sensor 212, is configured to authenticate the user beforethey even picks up the UE 102. This is illustrated in FIG. 3 , whichshows the user 120, with portions of their face that are used in facialrecognition (e.g., their chin, nose, or cheekbones) at an angle, whichcan be as little as ten degrees relative to plane 304 of the display116. Also shown, the user 120 is authenticated while having their facemore than one meter away from the face-unlock sensor 212, shown atfacial distance 306. By so doing, the techniques permit nearly seamlessand immediate authentication, even with the UE 102 oriented upside-downor at odd angles.

In more detail, consider FIG. 4 , which illustrates an exampleimplementation 400 of the UE 102 (including the radar manager 106, themovement manager 110, and the state manager 112) that can implementtechniques for authentication management through IMU and radar. The UE102 of FIG. 4 is illustrated with a variety of example devices,including a UE 102-1, a tablet 102-2, a laptop 102-3, a desktop computer102-4, a computing watch 102-5, computing spectacles 102-6, a gamingsystem 102-7, a home-automation and control system 102-8, and amicrowave 102-9. The UE 102 can also include other devices, such astelevisions, entertainment systems, audio systems, automobiles, drones,trackpads, drawing pads, netbooks, e-readers, home security systems, andother home appliances. Note that the UE 102 can be wearable,non-wearable but mobile, or relatively immobile (e.g., desktops andappliances).

The UE 102 includes an inertial measurement unit 408 as an example ofthe sensors 108 described above. Exemplary overall lateral dimensions ofthe UE 102 can be, for example, approximately eight centimeters byapproximately fifteen centimeters. Exemplary footprints of the radarsystem 104 can be even more limited, such as approximately fourmillimeters by six millimeters with antennas included. The requirementof such a limited footprint for the radar system 104, which is needed toaccommodate the many other desirable features of the UE 102 in such aspace-limited package combined with power and processing limitations,can lead to compromises in the accuracy and efficacy of radar-basedgesture-recognition, at least some of which can be overcome in view ofthe teachings herein.

The UE 102 also includes one or more computer processors 402 and one ormore computer-readable media 404, which includes memory media andstorage media. Applications and/or an operating system (not shown)implemented as computer-readable instructions on the computer-readablemedia 404 can be executed by the computer processors 402 to provide someor all of the functionalities described herein, such as some or all ofthe functions of the radar manager 106, the movement manager 110, andthe state manager 112 (shown within the computer-readable media 404,though this is not required).

The UE 102 may also include a network interface 406. The UE 102 can usethe network interface 406 for communicating data over wired, wireless,or optical networks. By way of example and not limitation, the networkinterface 406 may communicate data over a local-area-network (LAN), awireless local-area-network (WLAN), a personal-area-network (PAN), awide-area-network (WAN), an intranet, the Internet, a peer-to-peernetwork, point-to-point network, or a mesh network.

In aspects, the radar system 104 is implemented at least partially inhardware. Various implementations of the radar system 104 can include aSystem-on-Chip (SoC), one or more Integrated Circuits (ICs), a processorwith embedded processor instructions or configured to access processorinstructions stored in memory, hardware with embedded firmware, aprinted circuit board with various hardware components, or anycombination thereof. The radar system 104 operates as a monostatic radarby transmitting and receiving its own radar signals. In someimplementations, the radar system 104 may also cooperate with otherradar systems 104 that are within an external environment to implement abistatic radar, a multi-static radar, or a network radar. Constraints orlimitations of the UE 102, however, may impact a design of the radarsystem 104. The UE 102, for example, may have limited power available tooperate the radar, limited computational capability, size constraints,layout restrictions, an exterior housing that attenuates or distortsradar signals, and so forth. The radar system 104 includes severalfeatures that enable advanced radar functionality and high performanceto be realized in the presence of these constraints, as furtherdescribed below.

Prior to setting out additional example ways in which the state manager112 may act, consider FIG. 5 , which illustrates the many information,power, and access states in which the UE 102 may operate, and which canbe managed by the state manager 112.

FIG. 5 illustrates access, information, and power states in which the UE102 may operate, each of which can be managed by the describedtechniques. These example levels and types of device states 500 areshown in three levels of granularity for visual brevity, though manylevels of each are contemplated for access state 502, power state 504,and information state 506. The access state 502 is shown with threeexamples levels of granularity, high-access state 502-1,intermediate-access state 502-2, and low-access state 502-3. Similarly,the power state 504 is shown three examples levels of granularity,high-power state 504-1, intermediate-power state 504-2, and low-powerstate 504-3. Likewise, the information state 506 is shown three exampleslevels of granularity, high-information state 506-1,intermediate-information state 506-2, and low-information state 506-3.

In more detail, the access state 502 is concerned with the access rightsavailable to a user of the device to the data, applications, andfunctions of the UE 102. This access can be high, sometimes referred toas an “unlocked” state for the UE 102. This high access level caninclude simply the applications and functions of the device, or may alsoinclude access to various accounts, such as bank accounts, social mediaaccounts, and so forth that are accessible through the UE 102. Manycomputing devices, such as the UE 102, require authentication to providehigh access, such as the high-access state 502-1. Various intermediatelevels of access (e.g., 502-2) can be permitted by the UE 102, however,such as a state permitting a user to access some but not all accounts,services, or components of the UE 102. Examples include allowing a userto take pictures but not to access previously-captured pictures. Otherexamples include allowing the user to answer a telephone call but notaccess a contact list when making a telephone call. These are but a fewof the many intermediate rights that the UE 102 can permit, shown withthe intermediate-access state 502-2. Lastly, the access state 502 canrefrain from permitting access, shown as the low-access state 502-3. Inthis case the device may be on, send notifications like an alarm to wakeup a user, and so forth, but not permit access to functions of the UE102 (or the UE 102 may simply be off, and thus permit no access).

The power state 504 is shown with three examples levels of granularity,the high-power state 504-1, the intermediate-power state 504-2, and thelow-power state 504-3. The power state 504 is concerned with an amountof power to one or more components of the UE 102, such as the radarsystem 104, the display 116, or other power-consuming components, suchas processors, cameras, microphone, voice assistant, touchscreen,sensors, radar, and components that are part of the authenticationsystem 114 (which may include the previous components listed as well).In the context of powering up a component, as well as the power states504 generally, the terms power, powering up, increasing power, reducingpower, and so forth can include, control of a power-managementintegrated circuit (PMIC); managing power rails extending from the PMIC;opening and closing switches between a power rail, the PMIC, and one ormore circuit components (e.g., the mentioned NIR components, cameras,displays, and radar); and providing a supply voltage to accurately andsafely operate a component, which may include ramping or distributing anapplied voltage or managing current in-rush.

Regarding the radar system 104, the power state 504 can be reduced bycollecting radar data at different duty cycles (e.g., lower frequenciesmay use less power and higher frequencies may use more power), turningvarious components off when the components are not active, or adjustinga power amplification level. By so doing, the radar system 104 may useapproximately 90 mW of power at the high-power state 504-1, 30 to 60 mWat the intermediate-power state 504-2, or less than 30 mW at thelow-power state 504-3 (e.g., the radar system 104 can operate from 2 to20 mW while still providing some usable radar data, such as userpresence). Each of these levels of power usage permit differentresolutions and distance. Additional details regarding power managementof the radar system 104 (and the UE 102) are described with reference toFIG. 6-1 .

In the context of altering states noted above, the state manager 112,based on the determinations by the radar manager 106 and the movementmanager 110, may power-up or power-down various components of the UE102.

For example, the state manager 112 can alter the power of theauthentication system 114 or the display 116 from a lower-power state(e.g., the low-power state 504-3 to the intermediate-power state 504-2or either of these to the high-power state 504-1). By so doing, the UE102 may more-quickly or more-easily engage with a user or authenticatethe user. Thus, the state manager 112 may alter the power-state 504 tobe a higher or lower power than is currently the case for that system ofthe UE 102 or for particular power-consuming entities associated withthe UE 102. Example components are described further as part of FIG. 2above, including powering up or down the face-unlock sensor 212 and itscomponents, the NIR flood illuminator 214 and the NIR dot projector 216,as well as the NIR cameras 218-1 and 218-2, reducing power to thesecomponents, a display, microphone, touch-input sensor, and so forth.

The third example state of the UE 102 is the information state 506,which is illustrated with the high-information state 506-1, theintermediate-information state 506-2, and the low-information state506-3. In more detail, the information state 506 is concerned with anamount of information provided to a user, e.g., the user 120 of FIG. 1 .In the context of notifications, the high-information state 506-1provides a highest level of information, and generally assumes that theUE 102 is unlocked or otherwise authenticated or has a user preferencefor providing high levels of information even without authentication.Examples include, for the high-information state 506-1, showing acaller's name, number, and even associated image when a call isreceived. Similarly, when a text or email is received, or other type ofmessage, the content is automatically presented through the display 116or audio speakers, a peripheral, and so forth. This assumes a high-levelof engagement, though a user's preferences can determine what engagementis required. Here it is assumed that there is some correlation betweenthe user's engagement and the amount of information provided, andtherefore, the techniques, by determining engagement, can tailor theinformation presented to that determination. Examples of reducedinformation, e.g., the intermediate-information state 506-2, includepresenting a ring tone when a call is received but not the caller'sname/identification, indicating that text message or email has beenreceived but only the subject line, or only the address, or part of thecontent in the body but not all of it, and so forth. The low-informationstate 506-3 presents little to no information that is personallyassociated with the user 120, but can include information that isgeneric or widely considered common knowledge or non-sensitive, such asthe display 116 showing a current date, time, weather condition,battery-power status, or that the UE 102 is on. Other examples of thelow-information state 506-3 include a blank or black screen when a textmessage is received with an audible “ping” indicating only that amessage has been received, or a ring tone for a call, but not the name,number, or other information about the caller.

FIG. 6-1 illustrates an example implementation 600 of the radar system104. In the example 600, the radar system 104 includes at least one ofeach of the following components: a communication interface 602, anantenna array 604, a transceiver 606, a processor 608, and a systemmedia 610 (e.g., one or more computer-readable storage media). Theprocessor 608 can be implemented as a digital signal processor, acontroller, an application processor, another processor (e.g., thecomputer processors 402 of the UE 102) or some combination thereof. Thesystem media 610, which may be included within, or be separate from, thecomputer-readable media 404 of the UE 102, includes one or more of thefollowing modules: an attenuation mitigator 614, a digital beamformer616, an angle estimator 618, a power-management module 620, or agesture-recognition module 621. These modules can compensate for, ormitigate the effects of, integrating the radar system 104 within the UE102, thereby enabling the radar system 104 to recognize small or complexgestures, distinguish between different orientations of the user (e.g.,“reach”), continuously monitor an external environment, or realize atarget false-alarm rate. With these features, the radar system 104 canbe implemented within a variety of different devices, such as thedevices illustrated in FIG. 4 .

Using the communication interface 602, the radar system 104 can provideradar data to the radar manager 106. The communication interface 602 maybe a wireless or wired interface based on the radar system 104 beingimplemented separate from, or integrated within, the UE 102. Dependingon the application, the radar data may include raw or minimallyprocessed data, in-phase and quadrature (I/Q) data, range-Doppler data,processed data including target location information (e.g., range,azimuth, elevation), clutter map data, and so forth. Generally, theradar data contains information that is usable by the radar manager 106for providing a user's intent to engage, disengage, or maintainengagement to the state manager 112.

The antenna array 604 includes at least one transmitting antenna element(not shown) and at least two receiving antenna elements (as shown inFIG. 7 ). In some cases, the antenna array 604 may include multipletransmitting antenna elements to implement a multiple-inputmultiple-output (MIMO) radar capable of transmitting multiple distinctwaveforms at a time (e.g., a different waveform per transmitting antennaelement). The use of multiple waveforms can increase a measurementaccuracy of the radar system 104. The receiving antenna elements can bepositioned in a one-dimensional shape (e.g., a line) or atwo-dimensional shape for implementations that include three or morereceiving antenna elements. The one-dimensional shape enables the radarsystem 104 to measure one angular dimension (e.g., an azimuth or anelevation) while the two-dimensional shape enables two angulardimensions to be measured (e.g., both azimuth and elevation). Exampletwo-dimensional arrangements of the receiving antenna elements arefurther described with respect to FIG. 7 .

FIG. 6-2 illustrates an example transceiver 606 and processor 608. Thetransceiver 606 includes multiple components that can be individuallyturned on or off via the power-management module 620 in accordance withan operational state of the radar system 104. Note that thepower-management module 620 can be separate, integrated with, or underthe control of the state manager 112, such as in cases where the statemanager 112 is powering up or down components (e.g., the authenticationsystem 114) used to authenticate a user. The transceiver 606 is shown toinclude at least one of each of the following components: an activecomponent 622, a voltage-controlled oscillator (VCO) andvoltage-controlled buffer 624, a multiplexer 626, an analog-to-digitalconverter (ADC) 628, a phase lock loop (PLL) 630, and a crystaloscillator 632. If turned on, each of these components consume power,even if the radar system 104 is not actively using these components totransmit or receive radar signals. The active component 622, forexample, can include an amplifier or filter that is coupled to a supplyvoltage. The VCO 624 generates a frequency-modulated radar signal basedon a control voltage that is provided by the PLL 630. The crystaloscillator 632 generates a reference signal for signal generation,frequency conversion (e.g., upconversion or downconversion), or timingoperations within the radar system 104. By turning these components onor off, the power-management module 620 enables the radar system 104 toquickly switch between active and inactive operational states andconserve power during various inactive time periods. These inactive timeperiods may be on the order of microseconds (μs), milliseconds (ms), orseconds (s).

The processor 608 is shown to include multiple processors that consumedifferent amounts of power, such as a low-power processor 608-1 and ahigh-power processor 608-2. As an example, the low-power processor 608-1can include a processor that is embedded within the radar system 104 andthe high-power processor can include the computer processor 402 or someother processor that is external to the radar system 104. Thedifferences in power consumption can result from different amounts ofavailable memory or computational ability. For instance, the low-powerprocessor 608-1 may utilize less memory, perform fewer computations, orutilize simpler algorithms relative to the high-power processor 608-2.Despite these limitations, the low-power processor 608-1 can processdata for less-complex radar-based applications, such as proximitydetection or motion detection (based on radar data rather than inertialdata). The high-power processor 608-2, in contrast, may utilize a largeamount of memory, perform a large amount of computations, or executecomplex signal processing, tracking, or machine-learning algorithms. Thehigh-power processor 608-2 may process data for high-profile radar-basedapplications, such as gesture-recognition, facial recognition (for theauthentication system 114), and provide accurate, high-resolution datathrough the resolution of angular ambiguities or distinguishing ofmultiple users and features thereof.

To conserve power, the power-management module 620 can control whetherthe low-power processor 608-1 or the high-power processor 608-2 are usedto process the radar data. In some cases, the low-power processor 608-1can perform a portion of the analysis and pass data onto the high-powerprocessor 608-2. Example data may include a clutter map, raw orminimally processed radar data (e.g., in-phase and quadrature data orrange-Doppler data), or digital beamforming data. The low-powerprocessor 608-1 may also perform some low-level analysis to determinewhether there is anything of interest in the environment for thehigh-power processor 608-2 to analyze. In this way, power can beconserved by limiting operation of the high-power processor 608-2 whileutilizing the high-power processor 608-2 for situations in whichhigh-fidelity or accurate radar data is requested by the radar-basedapplication. Other factors that can impact power consumption within theradar system 104 are further described with respect to FIG. 6-1 .

The gesture-recognition model 621 interprets gestures, such astouch-independent gestures, from radar data obtained by the radar system104. The gestures can be two-dimensional gestures (e.g., performed neara surface where the radar system 104 outputs microwave emissions). Thegestures may be three-dimensional gestures performed in the air.

Based on radar data, the gesture-recognition model 621 identifies cues,shapes and signs a user makes with their body, including their fingers,hands, eyes, head, mouth, etc. The gesture-recognition model 621 matchesthe user's movements to matching shapes, signs, and movements ofpredetermined gestures. In response to determining that the radar datamatches a particular gesture, the gesture-recognition model 621 outputsan indication of the gesture to other components to perform a function,for example, to control an operating system or application function suchas authenticating the user 120.

The gesture-recognition model 621 may be a machine-learned model, suchas a neural network, that is trained to identify touch-independentgestures from radar data. For example, the gesture-recognition model 621may be trained using training data that includes samples of radar dataand corresponding portions of gestures that match the radar data. Basedon the training data, the gesture-recognition model 621 determines rulesto apply to samples of radar data received by the radar system 104 sothat when similar radar data is received, the corresponding portions ofgestures are identified and used to construct a gesture prediction. Inexecuting the rules, the gesture-recognition model 621 can output anindication of a recognized gesture predicted from the radar data.

In some cases, the indication of the recognized gesture may beaccompanied by a confidence or score. The confidence indicates a degreeof confidence the radar data 104 has applied to the identified gesture.The gesture-recognition model 621 may adjust the confidence in anidentified gesture based on the context. For instance, thegesture-recognition model 621 applies a high-confidence when detectinggestures in an environment where the user is not moving as opposed to alow-confidence when detecting similar gestures in an environment wherethe user is moving. The gesture-recognition model 621 may apply alow-confidence when detecting gestures in an environment where a largeobject is occluding the radar system 104 as opposed to a high-confidencewhen detecting similar gestures in an environment where the user isviewing the UE 102. An application or other component that relies on adetected gesture may discard or process the gesture depending on theconfidence or score associated with the gesture. The describedtechniques and systems may apply the confidence or score to gate thegesture, so the gesture is discarded and not used to perform a function.

These and other capabilities and configurations, as well as ways inwhich entities of FIGS. 1, 2, 4, and 6-9 act and interact, are set forthin greater detail below. These entities may be further divided,combined, and so on. The environment 100 of FIG. 1 and the detailedillustrations of FIG. 2 through FIG. 9 illustrate some of many possibleenvironments and devices capable of employing the described techniques.FIGS. 6-9 describe additional details and features of the radar system104. In FIGS. 6-9 , the radar system 104 is described in the context ofthe UE 102, but as noted above, the applicability of the features andadvantages of the described systems and techniques are not necessarilyso limited, and other embodiments involving other types of electronicdevices may also be within the scope of the present teachings.

FIG. 7 illustrates example arrangements 700 of receiving antennaelements 702. If the antenna array 604 includes at least four receivingantenna elements 702, for example, the receiving antenna elements 702can be arranged in a rectangular arrangement 704-1 as depicted in themiddle of FIG. 7 . Alternatively, a triangular arrangement 704-2 or anL-shape arrangement 704-3 may be used if the antenna array 604 includesat least three receiving antenna elements 702.

Due to a size or layout constraint of the UE 102, an element spacingbetween the receiving antenna elements 702 or a quantity of thereceiving antenna elements 702 may not be ideal for the angles at whichthe radar system 104 is to monitor. In particular, the element spacingmay cause angular ambiguities to be present that make it challenging forconventional radars to estimate an angular position of a target.Conventional radars may therefore limit a field of view (e.g., anglesthat are to be monitored) to avoid an ambiguous zone, which has theangular ambiguities, and thereby reduce false detections. For example,conventional radars may limit the field of view to angles betweenapproximately −45 degrees to 45 degrees to avoid angular ambiguitiesthat occur using a wavelength of 8 millimeters (mm) and an elementspacing of 6.5 mm (e.g., the element spacing being 90% of thewavelength). Consequently, the conventional radar may be unable todetect targets that are beyond the 45-degree limits of the field ofview. In contrast, the radar system 104 includes the digital beamformer616 and the angle estimator 618, which resolve the angular ambiguitiesand enable the radar system 104 to monitor angles beyond the 45-degreelimit, such as angles between approximately −90 degrees to 90 degrees,or up to approximately −180 degrees and 180 degrees. These angularranges can be applied across one or more directions (e.g., azimuthand/or elevation). Accordingly, the radar system 104 can realize lowfalse-alarm rates for a variety of different antenna array designs,including element spacings that are less than, greater than, or equal tohalf a center wavelength of the radar signal.

Using the antenna array 604, the radar system 104 can form beams thatare steered or un-steered, wide or narrow, or shaped (e.g., as ahemisphere, cube, fan, cone, or cylinder). As an example, the one ormore transmitting antenna elements (not shown) may have an un-steeredomnidirectional radiation pattern or may be able to produce a wide beam,such as the wide transmit beam 706. Either of these techniques enablethe radar system 104 to illuminate a large volume of space. To achievetarget angular accuracies and angular resolutions, however, thereceiving antenna elements 702 and the digital beamformer 616 can beused to generate thousands of narrow and steered beams (e.g., 3000beams, 7000 beams, or 9000 beams), such as the narrow receive beam 708.In this way, the radar system 104 can efficiently monitor the externalenvironment and accurately determine arrival angles of reflectionswithin the external environment.

Returning to FIG. 6-1 , the transceiver 606 includes circuitry and logicfor transmitting and receiving radar signals via the antenna array 604.Components of the transceiver 606 can include amplifiers, mixers,switches, analog-to-digital converters, filters, and so forth forconditioning the radar signals. The transceiver 606 can also includelogic to perform in-phase/quadrature (I/Q) operations, such asmodulation or demodulation. The transceiver 606 can be configured forcontinuous wave radar operations or pulsed radar operations. A varietyof modulations can be used to produce the radar signals, includinglinear frequency modulations, triangular frequency modulations, steppedfrequency modulations, or phase modulations.

The transceiver 606 can generate radar signals within a range offrequencies (e.g., a frequency spectrum), such as between 1 gigahertz(GHz) and 400 GHz, between 4 GHz and 100 GHz, or between 57 GHz and 63GHz. The frequency spectrum can be divided into multiple sub-spectrathat have a similar bandwidth or different bandwidths. The bandwidthscan be on the order of 500 megahertz (MHz), 1 GHz, 2 GHz, and so forth.As an example, different frequency sub-spectra may include frequenciesbetween approximately 57 GHz and 59 GHz, 59 GHz and 61 GHz, or 61 GHzand 63 GHz. Multiple frequency sub-spectra that have a same bandwidthand may be contiguous or non-contiguous may also be chosen forcoherence. The multiple frequency sub-spectra can be transmittedsimultaneously or separated in time using a single radar signal ormultiple radar signals. The contiguous frequency sub-spectra enable theradar signal to have a wider bandwidth while the non-contiguousfrequency sub-spectra can further emphasize amplitude and phasedifferences that enable the angle estimator 618 to resolve angularambiguities. The attenuation mitigator 614 or the angle estimator 618may cause the transceiver 606 to utilize one or more frequencysub-spectra to improve performance of the radar system 104, as furtherdescribed with respect to FIGS. 8 and 9 . Some embodiments of thetechniques are particularly advantageous, such as when the UE 102 is ahandheld smartphone, the radar signals are in the 57 Ghz-64 Ghz band, apeak effective isotropic radiated power (EIRP) is in the range of 10dBm-20 dBm (10 mW-100 mW), and an average power-spectral density isabout 13 dBm/MHz, which has been found to suitably address radiationhealth and co-existence issues while also providing a nicely-sized“bubble” of radar detection (e.g., at least one meter and often up to orexceeding two meters in extent) near-around the smartphone and the userwithin which the described methods for authentication management throughIMU and radar provided particularly good time-saving convenience whileconserving power.

A power-management module 620 manages power usage to balance performanceand power consumption. For example, the power-management module 620communicates with the radar manager 106 to cause the radar system 104 tocollect data using a predefined radar-power state. Each predefinedradar-power state can be associated with a particular framing structure,a particular transmit power level, or particular hardware (e.g., thelow-power processor 608-1 or the high-power processor 608-2 of FIG. 6-2). Adjusting one or more of these affects the radar system's 104 powerconsumption. Reducing power consumption, however, affects performance,such as a gesture-frame update rate and response delay, which aredescribed below.

FIG. 6-3 illustrates an example relationship between power consumption,a gesture-frame update rate 634, and a response delay. In graph 636,radar-power states 638-1, 638-2, and 638-3 are associated with differentlevels of power consumption and different gesture-frame update rates634. The gesture-frame update rate 634 represents how often the radarsystem 104 actively monitors the external environment by transmittingand receiving one or more radar signals. Generally speaking, the powerconsumption is proportional to the gesture-frame update rate 634. Assuch, higher gesture-frame update rates 634 result in larger amounts ofpower being consumed by the radar system 104.

In graph 636, the radar-power state 638-1 utilizes a smallest amount ofpower whereas the radar-power state 638-3 consumes a largest amount ofpower. As an example, the radar-power state 638-1 consumes power on theorder of a few milliwatts (mW) (e.g., between approximately 2 mW and 4mW) whereas the radar-power state 638-3 consumes power on the order ofseveral milliwatts (e.g., between approximately 6 mW and 20 mW). Interms of the gesture-frame update rate 634, the radar-power state 638-1uses an update rate that is on the order of a few hertz (e.g.,approximately 1 Hz or less than 5 Hz) while the radar-power state 638-3uses a gesture-frame update rate 634 that is on the order of tens ofhertz (e.g., approximately 20 Hz or greater than 10 Hz).

Graph 640 depicts a relationship between the response delay and thegesture-frame update rate 634 for the different radar-power states 638-1to 638-3. Generally speaking, the response delay isinversely-proportional to both the gesture-frame update rate 634 and thepower consumption. In particular, the response delay exponentiallydecreases while the gesture-frame update rate 634 increases. Theresponse delay associated with the radar-power state 638-1 may be on theorder of hundreds of milliseconds (ms) (e.g., 1000 ms or more than 200ms) while the response delay associated with the radar-power state 638-3may be on the order of several milliseconds (e.g., 50 ms or less than100 ms). For the radar-power state 638-2, the power consumption,gesture-frame update rate 634, and response delay are between that ofthe radar-power state 638-1 and the radar-power state 638-3. Forinstance, the radar-power state's 638-2 power consumption isapproximately 5 mW, the gesture-frame update rate is approximately 8 Hz,and the response delay is between approximately 100 ms and 200 ms.

Instead of operating at either the radar-power state 638-1 or theradar-power state 638-3, the power-management module 620 dynamicallyswitches between the radar-power states 638-1, 638-2, and 638-3 (andsub-states between each of these radar-power states 638) such that theresponse delay and the power consumption are managed together based onthe activity within the environment. As an example, the power-managementmodule 620 activates the radar-power state 638-1 to monitor the externalenvironment or detect an approaching user. Later in time, thepower-management module 620 activates the radar-power state 638-3 if theradar system 104 determines the user is showing an intent to engage ormay be starting to do so, or starting to perform a gesture. Differenttriggers may cause the power-management module 620 to switch between thedifferent radar-power states 638-1 through 638-3. Example triggersinclude motion or the lack of motion, appearance or disappearance of theuser, the user moving into or out of a designated region (e.g., a regiondefined by range, azimuth, or elevation), a change in velocity of amotion associated with the user, an intent to engage determined by theradar manager 106 (e.g., a “reach” though some intents to engage requireadditional power, such as facial feature tracking), or a change inreflected signal strength (e.g., due to changes in radar cross section).In general, the triggers that indicate a lower probability of the userinteracting with the UE 102 or a preference to collect data using alonger response delay may cause the radar-power state 638-1 to beactivated to conserve power.

In general, the power-management module 620 determines when and howpower can be conserved, and incrementally adjusts power consumption toenable the radar system 104 to operate within power limitations of theUE 102. In some cases, the power-management module 620 may monitor anamount of available power remaining and adjust operations of the radarsystem 104 accordingly (e.g., due to a low battery). For example, if theremaining amount of power is low, the power-management module 620 maycontinue operating in the radar-power state 638-1 instead of switchingto either of the radar-power states 638-2 or 638-3.

Each power state 638-1 to 638-3 can be associated with a particularframing structure. The framing structure specifies a configuration,scheduling, and signal characteristics associated with the transmissionand reception of the radar signals. In general, the framing structure isset up such that the appropriate radar data can be collected based onthe external environment. The framing structure can be customized tofacilitate collection of different types of radar data for differentapplications (e.g., proximity detection, feature recognition, or gesturerecognition). During inactive times throughout each level of the framingstructure, the power-management module 620 can turn off the componentswithin the transceiver 606 in FIG. 6-2 to conserve power. An exampleframing structure is further described with respect to FIG. 6-4 .

FIG. 6-4 illustrates an example framing structure 642. In the depictedconfiguration, the framing structure 642 includes three different typesof frames. At a top level, the framing structure 642 includes a sequenceof gesture frames 644, which can be in the active state or the inactivestate. Generally speaking, the active state consumes a larger amount ofpower relative to the inactive state. At an intermediate level, theframing structure 642 includes a sequence of feature frames (FF) 646,which can similarly be in the active state or the inactive state.Different types of feature frames include a pulse-mode feature frame 648(shown at the bottom-left of FIG. 6-4 ) and a burst-mode feature frame650 (shown at the bottom-right of FIG. 6-4 ). At a low level, theframing structure 642 includes a sequence of radar frames (RF) 652,which can also be in the active state or the inactive state.

The radar system 104 transmits and receives a radar signal during anactive radar frame (RF) 652. In some situations, the radar frames 652are individually analyzed for basic radar operations, such as search andtrack, clutter-map generation, user location determination, and soforth. Radar data collected during each active radar frame 652 can besaved to a buffer after completion of the radar frame 652 or provideddirectly to the processor 608 of FIG. 6-1 .

The radar system 104 analyzes the radar data across multiple radarframes 652 (e.g., across a group of radar frames 652 associated with anactive feature frame 646) to identify a particular feature associatedwith one or more gestures. Example types of features include aparticular type of motion, a motion associated with a particularappendage (e.g., a hand or individual fingers), and a feature associatedwith different portions of the gesture. To recognize a gesture performedby the user 120 during an active gesture frame 644, the radar system 104analyzes the radar data associated with one or more active featureframes 646.

Depending upon the type of gesture, a duration of the gesture frame 644may be on the order of milliseconds or seconds (e.g., betweenapproximately 10 ms and 10 s). After the active gesture frames 644occur, the radar system 104 is inactive, as shown by inactive gestureframes 644-3 and 644-4. A duration of the inactive gesture frames 644 ischaracterized by a deep sleep time 654, which may be on the order oftens of milliseconds or more (e.g., greater than 50 ms). In an exampleimplementation, the radar system 104 can turn off all of the componentswithin the transceiver 606 to conserve power during the deep sleep time654.

In the depicted framing structure 642, each gesture frame 644 includes Kfeature frames 646, where K is a positive integer. If the gesture frame644 is in the inactive state, all of the feature frames 646 associatedwith that gesture frame 644 are also in the inactive state. In contrast,an active gesture frame 644 includes J active feature frames 646 and K-Jinactive feature frames 646, where J is a positive integer that is lessthan or equal to K. A quantity of feature frames 646 can be based on acomplexity of the gesture and may include a few to a hundred featureframes 646 (e.g., K may equal 2, 10, 30, 60, or 100). A duration of eachfeature frame 646 may be on the order of milliseconds (e.g., betweenapproximately 1 ms and 50 ms).

To conserve power, the active feature frames 646-1 to 646-J occur priorto the inactive feature frames 646-(J+1) to 646-K. A duration of theinactive feature frames 646-(J+1) to 646-K is characterized by a sleeptime 656. In this way, the inactive feature frames 646-(J+1) to 646-Kare consecutively executed such that the radar system 104 can be in apowered-down state for a longer duration relative to other techniquesthat interleave the inactive feature frames 646-(J+1) to 646-K with theactive feature frames 646-1 to 646-J. Generally speaking, increasing aduration of the sleep time 656 enables the radar system 104 to turn offcomponents within the transceiver 606 that require longer start-uptimes.

Each feature frame 646 includes L radar frames 652, where L is apositive integer that may or may not be equal to J or K. In someimplementations, a quantity of radar frames 652 may vary acrossdifferent feature frames 646 and may comprise a few frames or hundredsof frames (e.g., L may equal 5, 15, 30, 100, or 500). A duration of aradar frame 652 may be on the order of tens or thousands of microseconds(e.g., between approximately 30 μs and 5 ms). The radar frames 652within a particular feature frame 646 can be customized for apredetermined detection range, range resolution, or Doppler sensitivity,which facilitates detection of a particular feature and gesture. Forexample, the radar frames 652 may utilize a particular type ofmodulation, bandwidth, frequency, transmit power, or timing. If thefeature frame 646 is in the inactive state, all of the radar frames 652associated with that feature frame 646 are also in the inactive state.

The pulse-mode feature frame 648 and the burst-mode feature frame 650include different sequences of radar frames 652. Generally speaking, theradar frames 652 within an active pulse-mode feature frame 648 transmitpulses that are separated in time by a predetermined amount. Incontrast, the radar frames 652 within an active burst-mode feature frame650 transmit pulses continuously across a portion of the burst-modefeature frame 650 (e.g., the pulses are not separated by a predeterminedamount of time).

Within each active pulse-mode feature frame 648, the sequence of radarframes 652 alternates between the active state and the inactive state.Each active radar frame 652 transmits a radar signal (e.g., chirp),which is illustrated by a triangle. A duration of the radar signal ischaracterized by an active time 658. During the active time 658, thecomponents within the transceiver 606 are powered-on. During ashort-idle time 660, which includes the remaining time within the activeradar frame 652 and a duration of the following inactive radar frame652, the radar system 104 conserves power by turning off componentswithin the transceiver 606 that have a start-up time within a durationof the short-idle time 660.

An active burst-mode feature frame 650 includes M active radar frames652 and L-M inactive radar frames 652, where M is a positive integerthat is less than or equal to L. To conserve power, the active radarframes 652-1 to 652-M occur prior to the inactive radar frames 652-(M+1)to 652-L. A duration of the inactive radar frames 652-(M+1) to 652-L ischaracterized by a long-idle time 662. By grouping the inactive radarframes 652-(M+1) to 652-L together, the radar system 104 can be in apowered-down state for a longer duration relative to the short-idle time660 that occurs during the pulse-mode feature frame 648. Additionally,the power management module 620 can turn off additional componentswithin the transceiver 606 that have start-up times that are longer thanthe short-idle time 660 and shorter that the long-idle time 662.

Each active radar frame 652 within an active burst-mode feature frame650 transmits a portion of a radar signal. In this example, the activeradar frames 652-1 to 652-M alternate between transmitting a portion ofthe radar signal that increases in frequency and a portion of the radarsignal that decreases in frequency.

The framing structure 642 enables power to be conserved throughadjustable duty cycles within each frame type. A first duty cycle 664 isbased on a quantity of active feature frames 646 (J) relative to a totalquantity of feature frames 646 (K). A second duty cycle 665 is based ona quantity of active radar frames 652 (e.g., L/2 or M) relative to atotal quantity of radar frames 652 (L). A third duty cycle 668 is basedon a duration of the radar signal relative to a duration of a radarframe 652.

Consider an example framing structure 642 for the power state 638-1 thatconsumes approximately 2 mW of power and has a gesture-frame update rate634 between approximately 1 Hz and 4 Hz. In this example, the framingstructure 642 includes a gesture frame 644 with a duration betweenapproximately 250 ms and 1 second. The gesture frame 644 includesthirty-one pulse-mode feature frames 648 (e.g., L is equal to 31). Oneof the thirty-one pulse-mode feature frames 648 is in the active state.This results in the duty cycle 664 being approximately equal to 3.2%. Aduration of each pulse-mode feature frame 648 is between approximately 8ms and 32 ms. Each pulse-mode feature frame 648 is composed of eightradar frames 652. Within the active pulse-mode feature frame 648, alleight radar frames 652 are in the active state. This results in the dutycycle 665 being equal to 100%. A duration of each radar frame 652 isbetween approximately 1 ms and 4 ms. An active time 658 within each ofthe active radar frames 652 is between approximately 32 μs and 128 μs.As such, the resulting duty cycle 668 is approximately 3.2%. Thisexample framing structure 642 has been found to yield good performanceresults. These good performance results are in terms of good gesturerecognition and presence detection while also yielding good powerefficiency results in the application context of a handheld smartphonein a low-power state (e.g., low-power state 504-3).

Based on the framing structure 642, the power management module 620 candetermine a time for which the radar system 104 is not activelycollecting radar data. Based on this inactive time period, the powermanagement module 620 can conserve power by adjusting an operationalstate of the radar system 104 and turning off one or more components ofthe transceiver 606, as further described below.

As noted, the power-management module 620 can conserve power by turningoff one or more components within the transceiver 606 (e.g., avoltage-controlled oscillator, a multiplexer, an analog-to-digitalconverter, a phase lock loop, or a crystal oscillator) during inactivetime periods. These inactive time periods occur if the radar system 104is not actively transmitting or receiving radar signals, which may be onthe order of microseconds (μs), milliseconds (ms), or seconds (s).Further, the power-management module 620 can modify transmission powerof the radar signals by adjusting an amount of amplification provided bya signal amplifier. Additionally, the power-management module 620 cancontrol the use of different hardware components within the radar system104 to conserve power. If the processor 608 comprises a lower-powerprocessor and a higher-power processor (e.g., processors with differentamounts of memory and computational capability), for example, thepower-management module 620 can switch between utilizing the lower-powerprocessor for low-level analysis (e.g., detecting motion, determining alocation of a user, or monitoring the environment) and the higher-powerprocessor for situations in which high-fidelity or accurate radar datais requested by the radar manager 106 (e.g., for implementing thehigh-power state 504-1 of the authentication system 114 forauthenticating a user using radar data).

In addition to the internal power-saving techniques described above, thepower-management module 620 can also conserve power within the UE 102 byactivating or deactivating other external components or sensors that arewithin the UE 102, either alone or at a command of the authenticationsystem 114. These external components may include speakers, a camerasensor, a global positioning system, a wireless communicationtransceiver, a display, a gyroscope, or an accelerometer. Because theradar system 104 can monitor the environment using a small amount ofpower, the power-management module 620 can appropriately turn theseexternal components on or off based on where the user is located or whatthe user is doing. In this way, the UE 102 can seamlessly respond to theuser and conserve power without the use of automatic shut-off timers orthe user physically touching or verbally controlling the UE 102.

FIG. 8 illustrates additional details of an example implementation 800of the radar system 104 within the UE 102. In the example 800, theantenna array 604 is positioned underneath an exterior housing of the UE102, such as a glass cover or an external case. Depending on itsmaterial properties, the exterior housing may act as an attenuator 802,which attenuates or distorts radar signals that are transmitted andreceived by the radar system 104. The attenuator 802 may includedifferent types of glass or plastics, some of which may be found withindisplay screens, exterior housings, or other components of the UE 102and have a dielectric constant (e.g., relative permittivity) betweenapproximately four and ten. Accordingly, the attenuator 802 is opaque orsemi-transparent to a radar signal 806 and may cause a portion of atransmitted or received radar signal 806 to be reflected (as shown by areflected portion 804). For conventional radars, the attenuator 802 maydecrease an effective range that can be monitored, prevent small targetsfrom being detected, or reduce overall accuracy.

Assuming a transmit power of the radar system 104 is limited, andre-designing the exterior housing is not desirable, one or moreattenuation-dependent properties of the radar signal 806 (e.g., afrequency sub-spectrum 808 or a steering angle 810) orattenuation-dependent characteristics of the attenuator 802 (e.g., adistance 812 between the attenuator 802 and the radar system 104 or athickness 814 of the attenuator 802) are adjusted to mitigate theeffects of the attenuator 802. Some of these characteristics can be setduring manufacturing or adjusted by the attenuation mitigator 614 duringoperation of the radar system 104. The attenuation mitigator 614, forexample, can cause the transceiver 606 to transmit the radar signal 806using the selected frequency sub-spectrum 808 or the steering angle 810,cause a platform to move the radar system 104 closer or farther from theattenuator 802 to change the distance 812, or prompt the user to applyanother attenuator to increase the thickness 814 of the attenuator 802.

Appropriate adjustments can be made by the attenuation mitigator 614based on pre-determined characteristics of the attenuator 802 (e.g.,characteristics stored in the computer-readable media 404 of the UE 102or within the system media 610) or by processing returns of the radarsignal 806 to measure one or more characteristics of the attenuator 802.Even if some of the attenuation-dependent characteristics are fixed orconstrained, the attenuation mitigator 614 can take these limitationsinto account to balance each parameter and achieve a target radarperformance. As a result, the attenuation mitigator 614 enables theradar system 104 to realize enhanced accuracy and larger effectiveranges for detecting and tracking the user that is located on anopposite side of the attenuator 802. These techniques providealternatives to increasing transmit power, which increases powerconsumption of the radar system 104, or changing material properties ofthe attenuator 802, which can be difficult and expensive once a deviceis in production.

FIG. 9 illustrates an example scheme 900 implemented by the radar system104. Portions of the scheme 900 may be performed by the processor 608,the computer processors 402, or other hardware circuitry. The scheme 900can be customized to support different types of electronic devices andradar-based applications (e.g., the radar manager 106), and also enablesthe radar system 104 to achieve target angular accuracies despite designconstraints.

The transceiver 606 produces raw data 902 based on individual responsesof the receiving antenna elements 702 to a received radar signal. Thereceived radar signal may be associated with one or more frequencysub-spectra 904 that were selected by the angle estimator 618 tofacilitate angular ambiguity resolution. The frequency sub-spectra 904,for example, can be chosen to reduce a quantity of sidelobes or reducean amplitude of the sidelobes (e.g., reduce the amplitude by 0.5 dB, 1dB, or more). A quantity of frequency sub-spectra can be determinedbased on a target angular accuracy or computational limitations of theradar system 104.

The raw data 902 contains digital information (e.g., in-phase andquadrature data) for a period of time, different wavenumbers, andmultiple channels respectively associated with the receiving antennaelements 702. A Fast-Fourier Transform (FFT) 906 is performed on the rawdata 902 to generate pre-processed data 908. The pre-processed data 908includes digital information across the period of time, for differentranges (e.g., range bins), and for the multiple channels. A Dopplerfiltering process 910 is performed on the pre-processed data 908 togenerate range-Doppler data 912. The Doppler filtering process 910 maycomprise another FFT that generates amplitude and phase information formultiple range bins, multiple Doppler frequencies, and for the multiplechannels. The digital beamformer 616 produces beamforming data 914 basedon the range-Doppler data 912. The beamforming data 914 contains digitalinformation for a set of azimuths and/or elevations, which representsthe field of view for which different steering angles or beams areformed by the digital beamformer 616. Although not depicted, the digitalbeamformer 616 may alternatively generate the beamforming data 914 basedon the pre-processed data 908 and the Doppler filtering process 910 maygenerate the range-Doppler data 912 based on the beamforming data 914.To reduce a quantity of computations, the digital beamformer 616 mayprocess a portion of the range-Doppler data 912 or the pre-processeddata 908 based on a range, time, or Doppler frequency interval ofinterest.

The digital beamformer 616 can be implemented using a single-lookbeamformer 916, a multi-look interferometer 918, or a multi-lookbeamformer 920. In general, the single-look beamformer 916 can be usedfor deterministic objects (e.g., point-source targets having asingle-phase center). For non-deterministic targets (e.g., targetshaving multiple phase centers), the multi-look interferometer 918 or themulti-look beamformer 920 are used to improve accuracies relative to thesingle-look beamformer 916. Humans are an example of a non-deterministictarget and have multiple phase centers 922 that can change based ondifferent aspect angles, as shown at 924-1 and 924-2. Variations in theconstructive or destructive interference generated by the multiple phasecenters 922 can make it challenging for conventional radar systems toaccurately determine angular positions. The multi-look interferometer918 or the multi-look beamformer 920, however, perform coherentaveraging to increase an accuracy of the beamforming data 914. Themulti-look interferometer 918 coherently averages two channels togenerate phase information that can be used to accurately determine theangular information. The multi-look beamformer 920, on the other hand,can coherently average two or more channels using linear or non-linearbeamformers, such as Fourier, Capon, multiple signal classification(MUSIC), or minimum variance distortion-less response (MVDR). Theincreased accuracies provided via the multi-look beamformer 920 or themulti-look interferometer 918 enable the radar system 104 to recognizesmall gestures or distinguish between multiple portions of the user(e.g., facial features).

The angle estimator 618 analyzes the beamforming data 914 to estimateone or more angular positions. The angle estimator 618 may utilizesignal processing techniques, pattern matching techniques, ormachine-learning. The angle estimator 618 also resolves angularambiguities that may result from a design of the radar system 104 or thefield of view the radar system 104 monitors. An example angularambiguity is shown within an amplitude plot 926 (e.g., amplituderesponse).

The amplitude plot 926 depicts amplitude differences that can occur fordifferent angular positions of the target and for different steeringangles 810. A first amplitude response 928-1 (illustrated with a solidline) is shown for a target positioned at a first angular position930-1. Likewise, a second amplitude response 928-2 (illustrated with adotted line) is shown for the target positioned at a second angularposition 930-2. In this example, the differences are considered acrossangles between −180 degrees and 180 degrees.

As shown in the amplitude plot 926, an ambiguous zone exists for the twoangular positions 930-1 and 930-2. The first amplitude response 928-1has a highest peak at the first angular position 930-1 and a lesser peakat the second angular position 930-2. While the highest peak correspondsto the actual position of the target, the lesser peak causes the firstangular position 930-1 to be ambiguous because it is within somethreshold for which conventional radars may be unable to confidentlydetermine whether the target is at the first angular position 930-1 orthe second angular position 930-2. In contrast, the second amplituderesponse 928-2 has a lesser peak at the second angular position 930-2and a higher peak at the first angular position 930-1. In this case, thelesser peak corresponds to the target's location.

While conventional radars may be limited to using a highest peakamplitude to determine the angular positions, the angle estimator 618instead analyzes subtle differences in shapes of the amplitude responses928-1 and 928-2. Characteristics of the shapes can include, for example,roll-offs, peak or null widths, an angular location of the peaks ornulls, a height or depth of the peaks and nulls, shapes of sidelobes,symmetry within the amplitude response 928-1 or 928-2, or the lack ofsymmetry within the amplitude response 928-1 or 928-2. Similar shapecharacteristics can be analyzed in a phase response, which can provideadditional information for resolving the angular ambiguity. The angleestimator 618 therefore maps the unique angular signature or pattern toan angular position.

The angle estimator 618 can include a suite of algorithms or tools thatcan be selected according to the type of UE 102 (e.g., computationalcapability or power constraints) or a target angular resolution for theradar manager 106. In some implementations, the angle estimator 618 caninclude a neural network 932, a convolutional neural network (CNN) 934,or a long short-term memory (LSTM) network 936. The neural network 932can have various depths or quantities of hidden layers (e.g., threehidden layers, five hidden layers, or ten hidden layers) and can alsoinclude different quantities of connections (e.g., the neural network932 can comprise a fully-connected neural network or apartially-connected neural network). In some cases, the CNN 934 can beused to increase computational speed of the angle estimator 618. TheLSTM network 936 can be used to enable the angle estimator 618 to trackthe target. Using machine-learning techniques, the angle estimator 618employs non-linear functions to analyze the shape of the amplituderesponse 928-1 or 928-2 and generate angular probability data 938, whichindicates a likelihood that the user or a portion of the user is withinan angular bin. The angle estimator 618 may provide the angularprobability data 938 for a few angular bins, such as two angular bins toprovide probabilities of a target being to the left or right of the UE102, or for thousands of angular bins (e.g., to provide the angularprobability data 938 for a continuous angular measurement).

Based on the angular probability data 938, a tracker module 940 producesangular position data 942, which identifies an angular location of thetarget. The tracker module 940 may determine the angular location of thetarget based on the angular bin that has a highest probability in theangular probability data 938 or based on prediction information (e.g.,previously-measured angular position information). The tracker module940 may also keep track of one or more moving targets to enable theradar system 104 to confidently distinguish or identify the targets.Other data can also be used to determine the angular position, includingrange, Doppler, velocity, or acceleration. In some cases, the trackermodule 940 can include an alpha-beta tracker, a Kalman filter, amultiple hypothesis tracker (MHT), and so forth.

A quantizer module 944 obtains the angular position data 942 andquantizes the data to produce quantized angular position data 946. Thequantization can be performed based on a target angular resolution forthe radar manager 106. In some situations, fewer quantization levels canbe used such that the quantized angular position data 946 indicateswhether the target is to the right or to the left of the UE 102 oridentifies a 90-degree quadrant the target is located within. This maybe sufficient for some radar-based applications, such as user proximitydetection. In other situations, a larger number of quantization levelscan be used such that the quantized angular position data 946 indicatesan angular position of the target within an accuracy of a fraction of adegree, one degree, five degrees, and so forth. This resolution can beused for higher-resolution radar-based applications, such asgesture-recognition, or in implementations of the attention state or theinteraction state as described herein. In some implementations, thedigital beamformer 616, the angle estimator 618, the tracker module 940,and the quantizer module 944 are together implemented in a singlemachine-learning module.

Among the advantages of the described implementations, includingimplementations in which radar is used to determine a user's intent toengage, disengage, or maintain engagement, and further includingimplementations in which radar is used to detect user action that iscategorized as an indication of a user intent to engage or interact withthe electronic device, either of which might alternatively be achievableusing the on-device camera that is provided with most modernsmartphones, is that the power usage of the radar system issubstantially less than the power usage of the camera system, while thepropriety of the results can often be better with the radar system thanwith the camera system. For example, using the radar system describedhereinabove, the desired user-intention detection can be achieved ataverage power ranging from single-digit milliwatts to just a few dozenmilliwatts (e.g., 10 mW, 20 mW, 30 mW or 40 mW), even including theprocessing power for processing the radar vector data to make thedeterminations. At these low levels of power, it would be readilyacceptable to have the radar system 104 enabled at all times. As such,for example, with the smartphone radar system in the always-enabledstate, the desired delightful and seamless experience presentlydescribed can still be provided for a user that has been sitting acrossthe room from their smartphone for many hours.

In contrast, the optical cameras provided with most of today'ssmartphones typically operate at hundreds of milliwatts of power (e.g.,an order of magnitude higher than 40 mW, which is 400 mW). At such powerrates, optical cameras would be disadvantageous because they wouldsignificantly reduce the battery life of most of today's smartphones, somuch so as to make it highly impractical, if not prohibitive, to havethe optical camera in an always-on state. An additional advantage of theradar system is that the field of view can be quite large, readilyenough to detect a user walking up from any direction even when lyingflat and face-up on a table (for many typical implementations in whichthe radar chip is facing outward in the same general direction as theselfie camera) and, furthermore, by virtue of its Doppler processingability can be highly effective (especially at operating frequenciesnear 60 GHz) in detecting even relatively subtle movements of movingbodies from the variety of directions.

Additionally, the radar system can operate in environments in which theperformance of the camera system is reduced or restricted. For example,in lower-light environments, the camera system may have a reducedability to detect shape or movement. In contrast, the radar systemperforms as well in lower light as in full light. The radar system canalso detect presence and gestures through some obstacles. For instance,if the smartphone is in a pocket or a jacket or pair of pants, a camerasystem cannot detect a user or a gesture. The radar system, however, canstill detect objects in its field, even through a fabric that wouldblock the camera system. An even further advantage of using a radarsystem over an onboard video camera system of a smartphone is privacy,because a user can have the advantages of the herein describeddelightful and seamless experiences while at the same time not needingto be worried that there is a video camera taking video of them for suchpurposes.

The entities of FIGS. 1, 2, 4, and 6-9 may be further divided, combined,used along with other sensors or components, and so on. In this way,different implementations of the UE 102, with different configurationsof the radar system 104 and the sensors 108, can be used to implementradar-based gesture-recognition with context-sensitive gating and othercontext-sensitive controls. The example operating environment 100 ofFIG. 1 and the detailed illustrations of FIGS. 2-9 illustrate but someof many possible environments and devices capable of employing thedescribed techniques.

Example Methods

This section illustrates example methods, which may operate separatelyor together in whole or in part. Various example methods are described,each set forth in a subsection for ease of reading; these subsectiontitles are not intended to limit the interoperability of each of thesemethods one with the other.

Authentication Management

FIG. 10 depicts an example method 1000 for managing authenticationthrough IMU and radar and is one example of managing power states for auser equipment. The method 1000 is shown as a set of blocks that specifyoperations performed but are not necessarily limited to the order orcombinations shown for performing the operations by the respectiveblocks. Further, any of one or more of the operations may be repeated,combined, reorganized, or linked to provide a wide array of additionaland/or alternate methods. In portions of the following discussion,reference may be made to the example operating environment 100 of FIG. 1or to entities or processes as detailed in other figures, reference towhich is made for example only. The techniques are not limited toperformance by one entity or multiple entities operating on one device.

At 1002, an intent to engage of a user is determined, based on radardata and by a user equipment, the intent to engage indicating that theuser intends to engage with the user equipment. As noted above, theintent to engage can be indicated by determining that the user 120 isreaching toward the UE 102, looking at the UE 102, or leaning toward ororienting their or their body toward the UE 102, to name just threeexamples.

At 1004, alternatively or in addition to the determination of the intentto engage through the radar data, a movement of the user equipment isdetermined based on inertial data. This movement can indicate the user's120 picking up the UE 102, touching the UE 102, and other movements asnoted above.

At 1006, responsive to the determination of the intent to engage and, insome cases, the determination of movement of the user equipment, a powerstate of a power-consuming component of an authentication system isaltered. The power state of the power-consuming component is alteredfrom a first power state to a second power state, the second power stateconsuming greater power than the first power state. This alteration canbe based on solely the intent to engage determined using the radar dataor also through the movement determined through the inertial data.Furthermore, the power state of the power-consuming component or otherpowered components can be further raised based on the movementdetermination. As noted above, this movement determination may confirmthe user's 120 intent to engage, also provide an intent to engage, orotherwise add speed and/or robustness to the determination to add power,resources, and so forth to the authentication system. Note that, in somecases, components of an authentication system remain powered even when auser has not been determined to be intending to engage. In such a case,the techniques act to perform an authentication process responsive tothe intent to engage being determined. In such a case latency is reducedeven if power is not conserved for that process. The techniques can,however, refrain from using resources not associated with theauthentication system, thereby conserving power in other ways.

The power state to which the power-consuming component of theauthentication system is altered may or may not be sufficient to enablethe authentication system to perform an authentication process on theuser. In some cases, the second power state of the power-consumingcomponent is not the high-power state 504-1. In such a case, the secondpower state is the intermediate-power state 504-2 as noted above. Thisintermediate-power state 504-2, in some cases, is sufficient forperformance of the power-consuming component, such as a camera thatincludes an intermediate-power state that is still capable of providingsensor data for authentication without fully powering up (e.g.,capturing an image of a user in full light rather than in darkness,etc.). Another example is the display 116, which can be powered toaccept touch input for a password without powering the display'sluminosity to full power. Another case includes the radar system 104,where at a fairly close range of a user's face to the radar system 104,full power is not required to provide sufficiently-accurate facialfeatures to the authentication system 114.

In some cases, the powering up of the component is an intermediate step,such as a warm-up sequence, that may prepare the component or simplyreduce latency by giving the component additional time. In such a case,the state manager 112 can determine not to proceed to high-power, suchas if an intent to disengage is determined prior to the component beingready to authenticate, the user 120 moving the UE 102 thereby preventingauthentication (e.g., into a pocket), and so forth. In some cases, thepowering is an intermediate step that is then fully powered responsiveto determining that the user 120 has moved the UE 102, illustrated at1004, and thus to a power sufficient to perform the authenticationprocess. This warm-up sequence powers the component to theintermediate-power state 504-2 and then, after some short period oftime, the component is powered sufficient to be used in theauthentication process (e.g., to the high-power state 504-1). In such acase, the component is at high-power (or nearly so) while in apost-warm-up sequence following the warm-up sequence. For componentsthat consume substantial power if left on when not needed, but alsorequire a noticeable amount of time to power-up, such as some infraredor near-infrared (IR, NIR) sensors, an intermediate-power state duringwhich a warm-up sequence is performed can save substantial power orreduce noticeable and potentially user-experience-damaging latency.

Example power-consuming components of an authentication system aredescribed above, such as face-unlock sensors 212 of the authenticationsystem 114 of FIG. 1 , a touchscreen of the display 116, the radarsystem 104, and the processor 608 (e.g., high-power processor 608-2).For specific details on the many potential power-consuming components ofa facial-recognition system for authentication, see FIG. 2 and itsdescription.

At 1008, an authentication process is performed by the authenticationsystem. In doing so, the authentication system 114 uses thepower-consuming component at the altered power state, such as the secondpower state or a third, higher-power state. The authentication processis effective to authenticate the user or determine that the user is notauthenticated, indicating that access to the UE 102 should not bepermitted. As noted, the authentication process can be through facialrecognition, finger-print reading, password or other credential entrythrough a touch or audio interface (e.g., touch-screen data-entrycomponent of the display 112), and so forth. The authentication processcompares identifying features of the user or credentials with somesecure storage of comparable features or credentials to determine theuser's identity as authentic, and thus permitted access to the UE 102.This can be as simple as comparing a six-digit password entered throughthe display's touch screen, or require greater computations and systemcomplexity, such as determining facial features based on sensor datareceived from the power-consuming component and comparing the determinedfacial features to a facial-feature library. While not required, thisfacial-feature library can be stored local to the UE 102 and createdduring a facial-feature initialization by the UE 102 with theauthentication system 114. Furthermore, this library can be securelystored at the UE 102, such as in the form of an embedding on a securechip integral with the UE 102. This is one way in which privacy of theuser 120 can be maintained.

Throughout this disclosure examples are described where a computingsystem (e.g., the UE 102, a client device, a server device, a computer,or other type of computing system) may analyze information (e.g., radar,inertial, and facial-recognition sensor data) associated with a user,such as the just-mentioned facial features at operation 1008. Thecomputing system, however, can be configured to only use the informationafter the computing system receives explicit permission from the user ofthe computing system to use the data. For example, in situations wherethe UE 102 analyzes sensor data for facial features to authenticate theuser 102, individual users may be provided with an opportunity toprovide input to control whether programs or features of the UE 102 cancollect and make use of the data. The individual users may have constantcontrol over what programs can or cannot do with sensor data. Inaddition, information collected may be pre-treated in one or more waysbefore it is transferred, stored, or otherwise used, so thatpersonally-identifiable information is removed. For example, before theUE 102 shares sensor data with another device (e.g., to train a modelexecuting at another device), the UE 102 may pre-treat the sensor datato ensure that any user-identifying information or device-identifyinginformation embedded in the data is removed. Thus, the user may havecontrol over whether information is collected about the user and theuser's device, and how such information, if collected, may be used bythe computing device and/or a remote computing system.

Returning to the method 1000, at 1010, alternatively or in addition, thepower state of a display is altered responsive to determining that theuser equipment has moved or is moving. This alteration can be to powerup a touch-input reception capability of the display or to simply changethe visual presentation of the display. One example includes addingluminosity to the display 116 so that, which a user touches the UE 102,the user sees that the UE 102 is aware of the user's intent and thus,presumably, is preparing to engage with the user 120. Similarly, the UE102 may do so responsive to the intent to engage determined at 1002.

In some cases, the authentication process is performed for some periodof time or iterations without success (e.g., some pre-set number or timeperiod). In such a case, the method 1000 can continue by re-performingthe authentication process or continue the process responsive to thedetermination of the movement at 1004, shown at 1012. This alternativeis shown with some of the dashed-line arrows in FIG. 10 .

At 1014, responsive to the authentication process of the user at 1008(or re-performance at 1012) being successful, the user is authenticated,and an access state of the UE 102 is altered. This alteration canincrease the access of the UE 102 to high-access state from a low, no,or intermediate-access state, and in such a case, the UE 102 is“unlocked.” This high-access state (e.g., the high-access state 502-1 ofFIG. 5 ) is not required, however. Some levels of authentication canreserve access, power, or information for subsequent authentication.Examples include authenticating the user for use of some but not all ofthe applications and/or accounts of the UE 102 (e.g., accounts topurchase music, bank accounts, etc.), and requiring additionalauthentication for those reserved access accounts and applications. Forexample, in addition to the high-access state 502-1, the state manager112 can cause the UE 102 to be placed in the high-information state506-1. Examples of this alteration to the information state includepresenting a last-engaged-with application or webpage, including at alast-engaged-with portion, such as on page four of a ten-page article ona webpage, or half-way into a song or video that reproduces where theuser 120 was last engaged or authenticated with the UE 102. The statemanager 112 may alter these states quickly and seamlessly, responsive toauthentication of the user 120.

By way of example, consider one embodiment of the application of method1000 to scenario 1100 illustrated in FIG. 11 . The scenario 1100includes five portions, each one chronologically following the priorportion. At a first portion of the scenario 1100, shown at scenarioportion 1100-1, a user 1102 is not looking at, touching, or otherwiseengaged with a smartphone 1104. Assume here that the smartphone 1104 isin low-access, low-power, and low-information states 501-3, 504-3, and506-3, respectively (e.g., the smartphone 1104 appears to be switchedoff, but has sufficient power to determine an intent to engage). Thisscenario portion 1100-1 is assumed to be the situation prior to theoperation of the method at 1002 in FIG. 10 . A second portion is shownat 1100-2, during which the user 1102 turns toward and looks at, butdoes not touch, the smartphone 1104. At this point, the techniques, atoperation 1002, determine, based on radar data, that the user 1102intends to engage with the smartphone 1104. This intent to engage isdetermined without use of a reach movement but is instead based on theuser 1102 looking toward and orienting their body toward the smartphone1104. The techniques make this determination through the radar manager106 at operation 1002, which passes the determination to the statemanager 112. Following this, the state manager 112, at operation 1006,alters a power state of a power-consuming component (the face-unlocksensor 212) of the authentication system 114. Note that this is donewell before the user reaches toward or picks up the smartphone 1104,reducing latency and causing the authentication system 114 to be readyto authenticate the user.

Assume also, that over the next half of a second, while thepower-consuming component is powering up, the user 1102 moves closer to,and reaches toward the smartphone 1104 (the reach shown with hand 1106).This is shown at a third portion 1100-3. At this point theauthentication system 114 performs an authentication process (operation1008) but assume that the authentication process is unsuccessful forsome number of iterations and/or a period of time. The techniques maycease the attempts to authenticate the user 1102, and thereby savepower. Here, however, as shown at portion 1100-4, the user 1102 touchesthe smartphone 1104. This is determined, at operation 1004, to bemovement of the smartphone 1104 through inertial data sensed by thesensors 108 of FIG. 1 . This movement determination is passed to thestate manager 112. Based on this movement, the state manager 112continues to cause the authentication system 114 to attempt toauthenticate the user 1102, as illustrated by operation 1012 of method1000. Further still, at the operation 1010, and also based on themovement, the state manager 112 illuminates a display 1108 of thesmartphone 1104. This illumination, or powering up of the display 1108,can be performed at the scenario portion 1100-2, 1100-3, or 11004, buthere is shown responsive to determining the user's 1102 touch of thesmartphone 1104 (shown with time and notification information at 1110).By so doing, the user 1102 is given feedback that the smartphone 1104 isaware that the user 1102 is intending to engage.

As noted, the state manager 112 causes the authentication system 114 tocontinue the authentication process and, through these continuedattempts, authenticates the user 1102. This is shown at portion 1100-5,resulting in the smartphone 1104 being at different states, high-access,high-power, and high-information states 502-1, 504-1, and 506-1,respectively, with the high-access state 502-1 shown with the display1108 presenting an unlock icon 1112. These state levels can be raisedautomatically by the state manager 112, providing a seamless userexperience for the user 1102.

In this example scenario 1100 the inertial data provided by the sensors108 causes the state manager 112 to ascertain, with a higher level ofconfidence and therefore justifying the additional power, that the user1102 intends to engage with the smartphone 1104 and therefore that theuser 1102 wants to be authenticated. This is but one example scenarioshowing how inertial data from an IMU and radar data from a radar systemcan be used to authenticate a user quickly, easily, and with reducedpower consumption.

Reducing High-Level States

FIG. 12 depicts an example method 1200 for reducing a high-level statethrough IMU and radar. The method 1200 is shown as a set of blocks thatspecify operations performed but are not necessarily limited to theorder or combinations shown for performing the operations by therespective blocks. Further, any of one or more of the operations may berepeated, combined, reorganized, or linked to provide a wide array ofadditional and/or alternate methods, including with other methods setforth in this document (e.g., methods 1000, 1400, 1700, and 1800). Inportions of the following discussion, reference may be made to theexample operating environment 100 of FIG. 1 or to entities or processesas detailed in other figures, reference to which is made for exampleonly. The techniques are not limited to performance by one entity ormultiple entities operating on one device.

Optionally, at 1202 and prior to operations 1204 or 1206, an inactivitytime period is determined to have expired. In contrast to some other,conventional techniques that rely solely on expiration of a time period,method 1200 may use or refrain from using an inactivity time period toreduce a high-level state for a user equipment. While this inactivitytimer is not required, use of a timer, even if a short-timer, in somecases saves power. In more detail, an inactivity timer starts when alast user action with a user equipment is received, such as when a lasttouch to a touch screen or button, audio command, or gesture input wasreceived by the user equipment. Note that while some conventionaltechniques use a timer solely, and because of this conventional timersoften last minutes (e.g., one, three, five, or ten minutes), the method1200 can use a time period that is relatively short, such as one half,one, three, five, ten, or twenty seconds. By so doing, the likelihood ofthe user equipment exposing information, making inappropriate accessavailable, and so forth is very low, while use of a short inactivitytime period can operate to save some amount of power by refraining fromperforming operations of 1204 and/or 1206 for the inactivity timeperiod.

At 1204, a movement is determined, during a high-level state of a userequipment during which a user is interacting or has recently interactedwith the user equipment. The movement manager 110 determines thismovement based on inertial data received from the sensors 108, which isintegral with the UE 102. As shown with the dashed-lined arrow, thisoperation can optionally be responsive to operation 1206 and/or 1202(not shown). This determined movement can be one or more of the variousmovements set forth above, such a movement indicating that the user 120is picking up the UE 102, walking with, placing down, putting in apocket or enclosure, or simply touching near to or touching the UE 102.In some cases, the movement manager 110 determines that a movement is oris not sufficient to alter a state of the UE 102, and thus pass to thestate manager 112. Examples include those noted above, such as notovercoming a threshold movement, those caused by ambient vibrations, andthose that, while movement, are not a sufficient change to an ongoingmovement. Thus, the movement manager 110 can determine that the UE 102is moving as the user 120 walks along with the UE 102, but that movementcan be determined not to be a change sufficient to indicate a potentialthat the user 120 may be disengaging from the UE 102. Another way tolook at this is that movement can be based on a change and not simply acurrent moving of the UE 102. Example changes include moving and thennot moving, such as a user walking with the UE 102 and placing it downon a table. While the inertial data from the sensors 108 might not catchthe user 120 placing the UE 102 on the table, the determination that theinertial data shows little to no movement when there was movementimmediately prior (the user 120 walking with the UE 102) may still bedetermined as movement at operation 1204 based on this immediately-priormovement.

In more detail, the techniques can tailor a user equipment's state tothe user's engagement. Thus, in some cases the user equipment is in ahigh-level state (or states) due to the user being highly engaged withthe user equipment. For example, the method 1200 may determine prior tooperations 1204 or 1206 that the user is interacting with the userequipment. This determination of the user's engagement can be based onprior radar data indicating an intent to engage by the user, based onaudio or touch input from the user, a command or input received from theuser and through the audio or touch sensor, a successful authenticationprocess, and so forth.

At 1206, an intent to disengage is determined based on radar data and bythe user equipment. The radar manager 106 receives radar data from theradar system 104 and, using this radar data, determines whether the userintends to disengage from the UE 102. This intent to disengage includesthe various types set forth above, such as a hand retraction of the user120 from the UE 102, a facial orientation change relative to the UE 102,the user 120 turning away from or orienting their or their back to theUE 102, and so forth.

As shown with the dashed-lined arrow, this operation 1206 can optionallybe responsive to operation 1204 (and/or 1202, not shown). In thesecases, the state manager 112 or the radar manager 106 acts to conservepower by refraining from determining the user's 120 intent to disengageuntil the movement is determined, and vice-versa for the movementdetermination at 1204. By so doing, power can be conserved. Thus, thepower-management module 620 can be directed by the techniques to keepthe radar system 104 at reduced power until the movement is determinedat 1204. Once movement is determined, the state manager 112 causes thepower-management module 620 to power-up the radar system 104 inpreparation to determine whether the user 120 is acting in a mannerindicating an intent to disengage.

At 1208, the high-level state of the user equipment is reduced to anintermediate-level or low-level state, responsive to the determinationof the movement and/or the intent to disengage. In more detail, see anexample high-level state 1208-1, which can be one or multiple statesinvolving access, power, or information, e.g., those illustrated in FIG.5 (the high-access state 502-1, the high-power 504-1, or thehigh-information state 506-1). The state manager 112, responsive todetermination of movement or an intent to disengage, or both, determinesto reduce one or more of the states of the UE 102. This is illustratedin FIG. 12 with arrows showing a reduction from the high-level 1208-1 toan intermediate level 1208-2 or a low-level 1208-3. These are but two ofvarious granularities of power, access, and information. As illustratedin FIG. 5 , the intermediate level 1208-2 and the low-level 1208-3include the intermediate-access state 502-2, the intermediate-powerstate 504-2, and the intermediate-information state 506-2, each of whichis described above. The low-level 1208-3 is illustrated with three lowstates, the low-access state 502-3, the low-power state 504-3, and thelow-information state 506-3. These states are described in detail above.Note that any one, two, or all three of these states can be reduced bythe state manager 112 at operation 1208, either each to a same level ordiffering levels. Thus, the state manager 112 may reduce the high-accessstate 502-1 to an intermediate or low state, and keep the power stateand the information state at high or a mix of levels. Similarly, thestate manager 112 may reduce the power state 504 to the low-power state504-3 while keeping the UE 102 at the high-access state 502-1 (e.g.,“unlocked”).

By way of example, consider the application of method 1200 to scenario1300 illustrated in FIG. 13 . The scenario 1300 includes three portions,each one chronologically following the prior portion. Prior to the firstportion of the scenario 1300, assume that user 1302 is actively engagedwith smartphone 1304 and that the smartphone 1304 is in high-levelstates, namely power, access, and information states. At the firstportion, shown at scenario portion 1300-1, the user 1302 walks up to atable, and places the smartphone 1304 on the table. At operation 1204,the sensors 108 receive inertial data either for the touching of thesmartphone 1304 on the table or a lack of inertial data when, previousto being placed on the table, inertial data indicated movement (based onthe user 1302 walking with the smartphone 1304). Based on either or bothof these inertial data, the movement manager 110 determines a movementfor the smartphone 1304 and passes this determination to the radarmanager 106 and/or the state manager 112.

Assume that the radar manager 106 provides the radar field 118 (notshown for visual brevity, see FIG. 1 for an example) either immediatelyresponsive to the movement data or was already doing so, and thereforereceives radar data indicating the user's 1302 body position and soforth. Based on this radar data, the radar manager 106 determines for afirst iteration (and likely multiple others) that, at operation 1206 forthe body, arm, and hand placement, the user 1302 is not intending todisengage at the scenario portion 1300-1. This is due to the user 1302having a body orientation toward the smartphone 1304 and the user's handand arm being oriented toward the smartphone 1304. Because of this, ahigh-information state 1306-1 is not altered.

At the scenario portion 1300-2, however, assume that roughly two secondslater, the user 1302 picks up their coffee cup and begins to walk awaywhile turning their body away from the smartphone 1304. At this point,the radar manager 106 determines that the user 1302 is intending todisengage from the smartphone 1304 based on the body orientation of theuser 1302 being turned partly away from the smartphone 1304, and theuser's 1302 arm and hand oriented toward the coffee cup and not thesmartphone 1304. The radar manager 106 passes this determination to thestate manager 112.

At operation 1208, responsive to receiving the movement and intent todisengage determinations, the state manager 112 reduces the informationstate of the smartphone 1304 from the high-information state 1306-1shown at scenario portion 1300-1 to the intermediate-information state1306-2. These example information states are shown with informationdisplayed at scenario portion 1300-1 showing content from two textmessages and a time of day. Immediately at the user 1302 turning theirbody and picking up their coffee cup, the information state is reducedto the intermediate-information state 1306-2, shown with the time of dayand reduced information about the text messages (shown with the name ofthe sender but no context). This intermediate amount of information canbe useful to the user 1302, as the user 1302 may change their mind aboutengaging, or want to look back at the smartphone 1304 to see if a newnotification has arrived, such as a text from a different person.

Also, or instead of showing the intermediate-information state 1306-2,and as part of operation 1208, the state manager 112 may proceed to alow level either immediately or after first being at an intermediatestate. Here assume that the state manager 112, responsive to additionaldeterminations by the radar manager 106 indicating that the user 1302intends to disengage or a higher confidence level thereof (e.g., hereshown with a high confidence as the user 1302 is now a few meters awayand has their back fully turned to the smartphone 1304), reduces theinformation state further to the low-information state 1306-3, shown asscenario portion 1300-3 presenting only a current time of day.

While this example shows changes to an information state, access andpower may also or instead be changed. This is shown in part with anunlock icon 1310 shown at scenario portion 1300-1, indicating a highlevel of access (e.g., the high-level access 502-1 of FIG. 5 ). At thescenario portion 1300-2 after the state manager 112 receives themovement data and the intent to disengage, the state manager 112 reducesthe access to a low level, which is indicated to the user with the lockicon 1312. Further still, power states can be altered, such as byreducing a luminosity of the smartphone's 1304 display (not shown) atthe scenario portions 1300-2 and/or 1300-3.

Maintaining an Authenticated State

FIG. 14 depicts an example method 1400 for maintaining an authenticatedstate. The method 1400 is shown as a set of blocks that specifyoperations performed but are not necessarily limited to the order orcombinations shown for performing the operations by the respectiveblocks. Further, any of one or more of the operations may be repeated,combined, reorganized, or linked to provide a wide array of additionaland/or alternate methods, including with other methods set forth in thisdocument (e.g., methods 1000, 1200, 1700, and 1800). In portions of thefollowing discussion, reference may be made to the example operatingenvironment 100 of FIG. 1 or to entities or processes as detailed inother figures, reference to which is made for example only. Thetechniques are not limited to performance by one entity or multipleentities operating on one device.

Prior to discussing method 1400, note that any of the methods describedabove, in whole or in part, can be combined with method 1400. Consider,for example, the performance of method 1000 in FIG. 10 . This method1000 describes one example of authentication management resulting inauthentication of a user. Responsive to this authentication, the userequipment enters into an authenticated state. This state is described ingreater detail above. Thus, the method 1000 (or some other manner ofauthentication of a user) is performed prior to method 1400.

At 1402, during an authenticated state of a user equipment, a potentialdisengagement by a user of the user equipment is determined. Thisdetermination of a potential disengagement by a user can includedetermining an intent to disengage by the user, as noted above, andother determinations set forth below. Also, as noted above, theauthenticated state permits access, by the user, of one or more of thedata, applications, functions, accounts, or components of the userequipment. Examples of an authenticated state include the high-accessstate 502-1 and the intermediate access state 502-2 noted in FIG. 5above. While either of these access states can be permitted by the UE102 when in the authenticated state (often based on a user preference oran operating system default setting), the authenticated state assumes aprevious authentication of the user. A user-selected preference orsetting, however, can permit a high or intermediate access of the UE 102without authentication. Thus, while the authenticated state may includeaccess permitted by the high and intermediate access states noted above,the high and intermediate access are not necessarily authenticatedstates.

As illustrated in FIG. 14 , determination of the potential disengagementcan be performed, optionally, responsive to (or through performing)operation 1404 or operation 1406, as well as other manners describedherein, such as through determining an intent to disengage at operation1206 of method 1200. At 1404, expiration of an inactivity time period isdetermined. As noted above, this inactivity time period can start when alast user action is received, an active engagement with the userequipment ends (or is last received), or when a last intent to engagewas determined. For example, an inactivity timer (e.g., a time period)begins when a user last touches a touch-sensitive display or button, alast-received audio command is spoken, or a last-determinedtouch-independent gesture (e.g., a gesture determined using the radarsystem 104 noted above) is performed.

At 1406, a movement of the user equipment is determined based oninertial data of an inertial measurement unit (IMU) integral with theuser equipment. Example movements and inertial data are described above,such as inertial data received from the sensors 108 of FIG. 1 . Thus, amovement determination is one way in which the method may determine thata user is potentially disengaging, such as by placing the UE 102 in alocker, bag, or pocket (though placing in a bag or pocket may later bedetermined to be a passive engagement, noted below).

At 1408, a passive engagement by the user with the user equipment isdetermined based on radar data. This determination of a passiveengagement can be responsive to determination at 1402 of the potentialdisengagement (shown with a dashed-line arrow), or it can be independentof, or coincident with, that determination. Performing operation 1408responsive to the determination of the potential disengagement can, insome cases, save power or reduce latency. For example, the method 1400may power-up components of the radar system 104 (see also FIGS. 6-1 and6-2 ) responsive to the determination of a potential disengagement. Thiscan save power as noted above or give additional time for the radarsystem 104 to prepare to determine whether the user is passively engagedwith the radar system 104.

In the context of FIG. 1 , the radar manager 106 determines that theuser 120 is passively engaged with the UE 102. This passive engagementcan be determined by the radar manager 106 in multiple ways, which canbe exclusive or overlap one with the other. For example, the radarmanager 106 can determine that the user is passively engaged based onthe radar data indicating that a hand of the user 120 is holding theuser equipment 102 at an orientation at which the display 116 of theuser equipment 102 is maintained. Thus, if the user 120 is holding theUE 102 steady (or steady enough to view content or permit another personto view content) the user 120 is passively engaged. Other examples ofdetermining passive engagement are described above, including the user120 looking at or orienting their or their body toward the UE 102.

Furthermore, the radar manager 106 can determine passive engagementbased on the radar data indicating that the user 120 is present, such asby being within two meters of the UE 102. Other distances can also orinstead be used, such as 1.5 meters, one meter, or even one half of onemeter. In effect, the radar manager 106 can determine that the user 120is passively engaged by being roughly within reach of the UE 102. Theradar manager 106 may do so explicitly by indicating that the user 120is passively engaged, or simply pass information indicating a distancefrom the UE 102, to the state manager 112. The state manager 112 thendetermines passive engagement based on the proximity of the user 120and, in some cases, context, such as other people (or lack thereof),whether or not the user 120 is in a vehicle (car, bus, train), at adesk, and so forth. A user sitting in their or their home, for example,may have a larger permitted distance than the user sitting in a crowdedcoffee shop or train.

At 1410, responsive to the determination of the passive engagement bythe user with the user equipment, the authenticated state is maintained.This maintaining of the authenticated state can continue until anotherpotential disengagement is determined, or for some time period, afterwhich method 1400 can again be performed. One example of anauthenticated state is the high-access state 502-1 of FIG. 5 . In manysituations this authenticated state is an unlock state for the UE 102,but in some other cases the authenticated state permits some but not allaccess to the UE 102, such as the above-described intermediate-accessstate 502-2.

This maintaining of the authenticated state for the UE 102 does notrequire that other states be maintained. For example, in cases where theuser 120 is within two meters of the UE 102, but may or may not belooking toward or oriented toward the UE 102, the state manager 112 canreduce a power state or information state of the UE 102, such as fromthe high-power state 504-1 and the high-information state 506-1 tointermediate or low power or information states noted in FIG. 5 . If,however, the passive engagement includes the user looking at the UE 102,the power or information states can also be maintained, such as tocontinue to present, through the display 116, content to the user 120.

Optionally, the method 1400 can proceed to operation 1412, in which apresence or an intent to engage of a non-user is determined based onradar data. This radar data can be the same or later-received radardata, such as radar data from the radar system 104 received some numberof seconds or minutes after the radar data on which the passiveengagement was based. Thus, at 1412 the radar manager 106 determinesthat a non-user is present or intends to engage with the UE 102. If anon-user, therefore, reaches for the UE 102, or looks at the display 116of the UE 102, the radar manager 106 can determine this presence orintent, and pass it to the state manager 112.

At 1414, responsive to the determination that the non-user is present orintends to engage with the user equipment, the maintenance of theauthenticated state is ceased. Thus, if a non-user walks up, reachesfor, or looks at the display 116 of the UE 102, the state manager 112ceases to maintain the authenticated state (or activelyde-authenticates) the UE 102. Along with this cessation, the statemanager 112 may also reduce other states, such as an information stateeffective to reduce or eliminate information presented to the non-user.Assume, for example, that an authenticated user is reading a privateemail on the subway train. If a person sitting behind their looks at thedisplay, possibly to read the private email, the state manager 112 canlock the UE 102 and cease to display the private email. This can beperformed quickly and seamlessly, further improving the privacy of auser.

At 1416, optionally after ceasing to maintain the authenticated state,the method can be returned to the authenticated state responsive to adetermination that the non-user is no longer present or no longerintending to engage. Continuing the example above, when the non-user inthe subway train looks away from the display 116 of the UE 102, thestate manager 112 may re-authenticate the user 120 through anauthentication process or simply by switching back to the authenticationstate without re-authenticating. Thus, the user 120 can simply go backto the previous states immediately on cessation of the condition thatcaused the de-authentication. While some authentication processes, suchas the system and process described herein, are both fast andpower-efficient, not performing an authentication process can be fasterand more-power-efficient. On returning to the authenticated state, thestate manager 112 can return the information state to the prior leveland at content matching the content last presented to the user 120. Inthis example, when the non-user looks away, the display 116 presents theprivate email at a same location last presented by the UE 102 to theuser 120. By so doing, seamless management of authentication andimproved information privacy is provided to users. Note that a selectionby the user 120 can override operations of the techniques, such as auser selection to de-authenticate. In some cases, the user 120 simplyturns off the UE 102, which is permitted by the methods describedherein.

Consider another example illustrated in FIG. 15 through a scenario 1500.The scenario 1500 includes four portions. At a first portion 1500-1,assume that a user 1502 has been authenticated to the smartphone 1504,such as through credential or facial-feature analysis, and thus that thesmartphone 1504 is in an authenticated state 1506. This authenticatedstate 1506 allows the user 1502 access to the smartphone 1504, which isshown through the user 1502 accessing content of the smartphone 1504 bywatching a television program about volcanic eruptions.

The scenario 1500 is shown diverging along two different paths. In onepath an inactivity timer begins when the user 120 ceases to touch orprovide input to the smartphone 1504, which here is when the user 120relaxes to watch the television program. In another case an inactivitytimer can begin or not, but a potential disengagement will be determinedwithout its expiration. Thus, at scenario portion 1500-2, after threeminutes of inactivity, the inactivity timer expires. Returning to FIG.14 , operation 1402 determines that a potential disengagement by theuser has occurred, due to the inactivity time period expiring atoperation 1404. For the second path shown at scenario portion 1500-3,operation 1402 determines that a potential disengagement by the user hasoccurred by determining, based on inertial data, that a movement of thesmartphone 1504 has occurred through performing operation 1406. Thecause of this movement is the user 1502 putting their foot on the edgeof the table on which the smartphone 1504 is resting.

The radar manager 106, responsive to either of these determinations of apotential disengagement, determines, based on radar data, that the user1502 is passively engaged with the smartphone 1504. This operation isperformed at 1408. Here assume that the user's 1502 presence or theirlooking at the smartphone 1504 are determined, either of which indicatesthat the user 1502 is passively engaged.

In response, at operation 1410, the state manager 112 maintains theauthenticated state. All of this can be performed seamlessly and withoutthe user 1502 noticing that it has been performed. As shown in scenarioportion 1500-4, the smartphone 1504 simply continues to present thetelevision program through either path.

Consider another scenario 1600 of FIG. 16 , which can follow thescenario 1500 or be an alternative, stand-alone scenario. The scenario1600 includes three scenario portions, in a first scenario portion1600-1, the user 1502 is watching the television program aboutvolcanoes, similarly, to as shown in FIG. 15 , here marked at content1602 of the smartphone 1504. The smartphone 1504 is in an authenticatedstate during this presentation of the program, such as the authenticatedstate 1506 noted in FIG. 15 .

At scenario portion 1600-2, however, a non-user 1604 sits down on thecouch with the user 1502. This non-user 1604 is a colleague of the user1502 and so the user 1502 turns their head and begins talking to thenon-user 1604. These actions of the user 1502 can be considered apotential disengagement, either turning their head or talking or both,as noted above. If considered a potential disengagement by the user1502, the state manager 112 reduces the state of the smartphone 1504,such as to reduce the access state or the information state, noted inFIGS. 5 and 12 (e.g., operations 1206 and 1208 of method 1200).

Assume, however, that the radar manager 106 determines, throughoperation 1412 of method 1400 and based on radar data, the presence ofthe non-user 1604. Based on this presence of the non-user 1604, thestate manager 112 ceases to maintain the authenticated state 1506 afterthe state manager 112 previously acted to maintain the authenticatedstate of the smartphone 1504 (e.g., through operation 1410 shown in FIG.15 ). Thus, the state manager 112 can cause the smartphone 1504 to bereduced to a non-authenticated state 1604, shown at an expanded view ofthe scenario portion 1600-2. This change is shown to the user 1502through a lock icon 1606, as well as by ceasing to present the content1602.

At scenario portion 1600-3, the non-user 1604 has left and the user 1502returns to looking at the smartphone 1504. The radar manager 106determines that the non-user 1604 is no longer present, indicates thisdetermination to the state manager 112, which then returns thesmartphone 1504 to the authenticated state 1506. Note that the statemanager 112 may also require a determination that the user 1502 isintending to engage with the smartphone 1504 or may simply return to theauthenticated state based on the non-user 1604 leaving the presence ofthe smartphone 1504. Note also that the techniques described in thisdocument can return a user to the spot at which they left off,seamlessly, thereby providing an excellent user experience. This isshown in FIG. 16 with the state manager 112 returning the smartphone1504 to a same television program and at a same or nearly a same pointthat was last presented to the user 1502. For some embodiments thetechniques allow the user, in a setup screen or similar deviceconfiguration screen, to dictate whether, at step 1416, the smartphone1504 will return to the authenticated state responsive to thedetermination that the non-user is no longer present or intending toengage, versus whether the smartphone 1504 will stay in anon-authenticated state until a more rigorous authentication processusing a power-consuming component of an authentication system (e.g.,step 1006, supra) is carried out. Stated differently, the techniques canprovide a user-selected setting, through a setup or similar deviceconfiguration, that causes the smartphone 1504 to remainde-authenticated once there has been the taint of a non-user, even ifthe taint is no longer there.

Gesture-Recognition Management

FIG. 17 depicts an example method 1700 for radar-basedgesture-recognition with context-sensitive gating and othercontext-sensitive controls. The method 1700 is shown as a set of blocksthat specify operations performed but are not necessarily limited to theorder or combinations shown for performing the operations by therespective blocks. Further, any of one or more of the operations may berepeated, combined, reorganized, or linked to provide a wide array ofadditional and/or alternate methods, e.g., methods 1000, 1200, 1400, and1800. In portions of the following discussion, reference may be made tothe example operating environment 100 of FIG. 1 or to entities orprocesses as detailed in other figures, reference to which is made forexample only. The techniques are not limited to performance by oneentity or multiple entities operating on one device.

At operation 1702, sensor data from a plurality of sensors 108 isreceived. For example, the proximity sensor 208 generates sensor datathat is indicative of proximity to an object (e.g., the radar system 104configured as a proximity sensor by operating in a proximity mode). TheIMU 408 can produce sensor data indicating movement and other sensors108 can generate other sensor data that is used to define a context.

The radar system 104 may be operable in a low-power, proximity mode(e.g., the low-power state 504-3) to generate sensor data of sufficientresolution and quality for detecting proximity. The radar system 104 canalso operate in a high-power, gesture-recognition mode (e.g., thehigh-power state 504-1) to generate improved sensor data relative to thesensor data produced in a proximity mode. In gesture-recognition mode,the sensor data generated by the radar system 104 is a higher-resolutionor greater quality than in proximity mode because the sensor data isused for more complex gesture-recognition tasks. The sensor datareceived from the plurality of sensors at operation 1702 may indicatecourse movement and proximity whereas the sensor data collected from theradar system 104 to perform radar-based gesture-recognition may indicatemore-precise movement, proximity, or occlusion.

The sensor data can indicate proximity as a binary measurement of anobject, or as a variable measurement further specifying closeness to theobject. Proximity can indicate whether the radar system 104 or otherpart of the UE 102 is occluded by the object (meaning that the user 120relative to the UE 102 or vice-versa are occluded by the object), agreater amount of presence indicating occlusion and a lesser amount ofpresence indicating little or no occlusion. The sensor data can definemovements effective to determine a position, speed, acceleration,velocity, rotation, orientation, or other movement or positioningcharacteristics of the UE 102.

At operation 1704, a context of the UE is determined. The sensor dataobtained at operation 1702 indicates an operating environment of the UE102 as the user 120 interacts with the UE 102. The sensor data mayinclude patterns or signatures indicating whether a movement isintentional or unintentional. The UE 102 can include, or access,machine-learned activity classifiers trained using machine learning torecognize patterns or signatures in sensor data that correspond toparticular user activities or device contexts. The machine-learnedactivity classifiers output notifications to applications and othersubscribers that use activity recognition to perform other tasks.Accelerations or vibrations identified in movement data correspond tosimilar vibrations and accelerations that the IMU 408 or other of thesensors 108 records as sensor data when the user 120 is walking orotherwise moving with the UE 102.

Recognized activities or movements can indicate different contexts.Examples of contexts include a walking context, a cycling context, adriving context, a riding context, or other activity contextcorresponding to a recognized activity. The movements can indicateposition and orientation, as well as movement or lack of movement,typically observed when the user 120 is viewing or holding the UE 102.Lack of movement can indicate a seated context, a stationary context, anunused context, or a stowed context. Opposite movements can correspondto opposite activities, for example, certain movements can indicate theuser is picking up the UE 102 and opposite or different movements canindicate the user putting the UE 102 down.

At operation 1706, whether the context satisfies requirements forradar-based gesture-recognition is determined. The UE 102 may determinethe context satisfies the requirements if sensor data from the sensors108 (e.g., the proximity sensor 208 and the IMU 408) overtime matchessensor data the UE 102 expects to detect when radar-basedgesture-recognition are typically received. The UE 102 may determine theopposite is true when the context does not satisfy the requirements forradar-based gesture-recognition and prevent or otherwise discardgestures recognized by the radar system 104.

For example, the UE 102 can condition radar-based gesture-recognition ona current context of the UE 102. The context satisfies the requirementsfor radar-based gesture-recognition if the user 120 is holding the UE102 while walking but not if the user 120 is not holding the UE 102while walking. Carrying the UE 102 in a pocket or backpack whilewalking, however, is not a context that satisfies the requirements forradar-based gesture-recognition (except in cases where gesture detectionthrough an intervening material is permitted).

A stowed context is when sensor data indicates the UE 102 is positionedin: a pocket of clothing worn by the user 120, a compartment in abackpack, briefcase, or suitcase, a storage bin in an airplane, taxi,automobile, boat, bus, or train, a console or glove box of a vehicle, orother enclosure. A holding or carrying context is identified when thesensor data indicates the user 120 is holding the UE 102. A stationarycontext is evident from sensor data indicating the UE 102 is not beingheld, motionless, or substantially not moving relative to a surface onwhich the UE 102 rests. A traveling context indicates the UE 102 ismoving, regardless whether the UE 102 is being held or stowed, forexample, if the user 120 is walking, driving, cycling, or otherwisemoving with the UE 102.

The context satisfies requirements of radar-based gesture-recognitionbased in part on whether the UE 102 is traveling, being held, or stowed.For example, being held and traveling is a context where the radarsystem 104 recognizes radar-based gestures, however, being stowed andtraveling may not be a context where the radar system 104 recognizesradar-based gestures.

The satisfaction of the context can further depend on orientation,specifically, carrying orientation. If the user 120 is walking andholding the UE 102, the context may still not satisfy the requirementsfor radar-based gesture-recognition if the user is not holding the UE120 in a particular way. For example, the user 120 holding the UE 102 inlandscape and/or in a portrait-down orientation (e.g., the touch screenof the UE 102 pointed near the ground) while the user 120 is walking maynot satisfy the requirements for radar-based gesture-recognition as theuser 120 is likely not wanting to interact with the UE 102 in thiscontext. Conversely, the user 120 holding the UE 102 in a differentorientation (e.g., portrait-up with the touch screen of the UE 102pointed towards the sky or the user's 120 face) while walking maysatisfy the requirements for radar-based gesture-recognition as the user120 is likely viewing the touchscreen of the UE 102 while walking.

The UE 102 can condition radar-based gesture-recognition on whether orhow the user 120 is holding the UE 102. For example, the contextsatisfies the requirements for radar-based gesture-recognition if theuser 120 is not holding the UE 102 while cycling or driving, such as ifthe UE 102 is in a stationary context, fixed to a mounting bracket on abike frame or attached to an automobile air vent or dash, while cyclingor driving. A similar cycling or driving context where the user isholding the UE 102, however, may not satisfy the requirements ofradar-based gesture-recognition.

Radar-based gesture-recognition can be conditioned by the UE 102 basedon occlusion from, or proximity to, an object. For example, in responseto detecting proximity to an object while the UE 102 is already instowed or stationary contexts, the radar system 104 enablesgesture-recognition model 621. The opposite may be true in response todetecting occlusion by an object while the UE 102 is already in stowedor stationary contexts, the radar system 104 disablesgesture-recognition model 621 in this case. For example, the UE 102placed face-up (touch-screen up) on a flat service may be a stationarycontext where proximity to an object or no occlusion is detected andtherefore gesture recognition is enabled. The UE 102 placed face-down(touch-screen down) on the flat surface is an opposite stationarycontext where occlusion is detected and therefore gesture recognition isgated.

Significant motion can condition the gesture-recognition model 621. Ifthe UE 102 is in a significant-motion context where the UE 102experiences frequent or strong movements or changes in movement, thecontext may be less suitable for radar-based gesture-detection. Forexample, if the user 120 carrying the UE 102 in their hand goes for arun, the radar system 104 gates the gesture-recognition model 621 toensure the radar system 104 does not incorrectly trigger anygesture-conditioned events.

The radar system 104 can apply different sensitivity levels fordifferent types of gestures or for different types of contexts. Contextswith significant motion may trigger gating for most radar-based gestureswhereas contexts with less motion may trigger gating only some of theradar-based gestures. As an example, the radar system 104 may recognizecourse-control (e.g., whole-hand) gestures in a high-vibrationmanufacturing context, however the same context may not be suitable forparticular fine-control (e.g., individual finger) radar-based gestureswhere the UE 102 or the user 120 is unsteady and moving. Rather thanattempt to recognize fine-control radar-based gestures in thehigh-vibration context, the radar system 104 gates thegesture-recognition feature for fine-control gestures while continuingto recognize course-control gestures in the same context. The radarsystem 104 applies a different sensitivity level to the fine-controlgestures so they get triggered more easily than the course-controlgestures do. The radar system 104 applies a different sensitivity levelto the course-control gestures to avoid being triggered as easily as thefine-control gestures do.

In a marine context, the user 120 interacts with the UE 102 as apassenger on a boat. The user 120 may hold the UE 102 or the UE 102 maybe a computing device built into the boat. The boat moves with oceanwaves. In a stormy environment, the radar system 104 can recognize thatcertain radar-based gestures may be hard to recognize when the UE 102 isundergoing large changes in pitch or orientation and gate theradar-based gestures rather than risk outputting a false-positive. Whenthe stormy environment calms and variations in pitch and orientationsubside, the radar system 104 automatically stops gating and enables theradar-based gestures that had been gated during the storm.

The radar system 104 can gate all radar-based gestures for a particularcontext, or only gate certain types of radar-based gestures. Forexample, for a reach-and-grab gesture, specifically reach-to-pick-up theUE 102, the radar system 104 can reduce false positives to theface-authentication system 114 by recognizing from the sensor data whenthe user 120 reaches and then picks up the UE 102 to trigger theface-authentication system 114. This, as opposed to triggering theface-authentication system 114 in response to recognizing just thereach. The radar system 104 can gate a radar-based gesture for answeringa telephone call when the sensor data indicates the UE 102 is in a quietor noisy environment, or an environment with an intermittentcommunications signal. The radar system 104 automatically un-gates andenables the telephone-answering radar-based gesture when the radarsystem 104 determines the UE 102 is in an office location or on a desknear a laptop computer where the user 120 is likely to want to answerthe telephone using the gesture-recognition model 621.

The UE 102 can condition radar-based gesture-recognition on whether thecontext indicates the user 120 is holding the UE 102, whether thecontext indicates the user 120 is walking or both. The context satisfiesthe requirements for touch-independent gesture-recognition if the UE 102determines the user 120 is holding the UE 102 and the user 120 iswalking. The context does not satisfy the requirements fortouch-independent gesture-recognition if the UE 102 determines the user120 is not holding the UE 102 and the user 120 is walking. The UE 102may determine the context satisfies the requirements if sensor data fromthe proximity sensor 208 and the IMU 408 overtime match sensor data theUE 102 expects to detect when the user 120 is walking and holding the UE102. The UE 102 may determine the opposite is true when the context doesnot satisfy the requirement and discard gestures recognized by the radarsystem 104.

In a reach-grab context, the user 120 reaches over the UE 102 as the UE102 is lying face-up on a table. The user 120 may be reaching to grabthe UE 102. The user 120 may be reaching to grab something beyond the UE102. Determining the context does not satisfy the requirements forradar-based gesture-recognition at 1706 can be in response todetermining the user 120 is not picking up the UE 102 after an objectcomes into proximity of the UE 102. If the user does not grab and pickup the UE 102 after reaching (e.g., the user 120 coming into proximity),the UE 102 gates the output from the gesture-recognition model 621(e.g., to prevent an authentication algorithm from executing aface-authentication) preventing a subscriber (e.g., an application, acomponent, a system service) from obtaining an indication of a gesture.Determining the context satisfies the requirements for radar-basedgesture-recognition at 1706 can be in response to determining the user120 is picking up the UE 102 after an object comes into proximity of theUE 102. If the user does grab and pick up the UE 102 after reaching, theUE 102 enables the output from the gesture-recognition model 621 (e.g.,to enable the authentication algorithm from executing theface-authentication) enabling the subscriber to obtain the indication ofthe gesture. Using context-sensitive gating and other context-sensitivecontrols in this way reduce reach-and-grab false-positives.

Other of the sensors 108, such as ambient light sensors, barometers,location sensors, optical sensors, infrared sensors, or the like, canprovide signals to the UE 102 to further define the context of the UE102 to improve gesture-recognition and other described techniques. At1706, determining the context satisfies the requirements for radar-basedgesture-recognition can be in response to location information, time ofday, barometric pressure, ambient light, ambient audio, and other sensorinformation for defining a context for gating or not-gating the radarsystem 104. For example, a context that specifies the UE 102 as beingnear the location of a movie theatre and in low-light conditions, whiledetecting loud and frequent ambient noises, is not a context suitablefor radar-based gesture-recognition. Whereas, near a rail station, whiledetecting low-lighting conditions and loud ambient noises, is a contextsuitable for radar-based (e.g., touch-independent) gesture-recognition.

At operation 1706, when the context does not satisfy requirements forradar-based gesture-recognition with the radar system, radar dataobtained by the radar system is gated and the method proceeds to B(described below in the description of FIG. 18 ). When the contextsatisfies requirements for radar-based gesture-recognition with a radarsystem at operation 1706, the radar data obtained by the radar system isinput to a model that determines radar-based gestures from the inputtedradar data at operation 1708.

At 1708, inputting the radar data obtained by the radar system 104 intothe gesture-recognition model 621 causes the gesture-recognition model621 to perform gesture-recognition techniques. The radar system 104 mayoperate in a high-power gesture-recognition mode for obtaining radardata that is of sufficient resolution, frequency, detail, and qualityfor radar-based (e.g., touch-independent) gesture-recognition. The radarsystem 104 may further operate in other modes, including a proximitymode, or a standby-mode. If multiple-mode-operations are supported, theradar system 104 can continue to operate in one or more modes, even if adifferent mode is disabled. For instance, disabling radar-basedgesture-recognitions may have no impact on radar-basedcollision-avoidance operations performed by the radar system 104. Someexamples of the radar system 104 may not be multimodal and therefore,disabling radar-based gesture-recognition can disable the radar system104 in its entirety.

In addition to being context-sensitive, the gesture-recognition model621 may adjust gating sensitivity based on identity of a subscriber. Thesubscriber can be an application, service, or component that receivesthe output from the gesture-recognition model 621. For example, thegesture-recognition model 621 provides an interface from which anapplication or component of the UE 102 (e.g., the authentication system114, an operating system function or service, an application, a driver)registers with the gesture-recognition model 621 and is assigned anidentity. The subscriber indicates a gating-sensitivity to apply fordifferent contexts. The subscriber may indicate a type of gesture ortype of radar-based gesture to apply the gating. For example, anoperating system may provide access to a function through a widget on alock screen user interface of the UE 102. The widget may recognizeradar-based gestures and may subscribe to the gesture-recognition outputfrom the gesture-recognition model 621. In some contexts, the outputfrom the gesture-recognition model 621 is gated to prevent an indicationof a gesture from being used by the subscriber. The output from thegesture-recognition model 621 is permitted in other contexts and anindication of the gesture is sent to the subscriber. In some contexts,the output from the gesture-recognition model 621 can be gated for onesubscriber but not gated for another. For example, that samegesture-recognition that is used by the widget-subscriber in aparticular context may be unusable by a different subscriber that electsto gate gestures for that context. A face-authentication application forexample may be unable to use the gesture information in certainconditions but the widget on the lock screen can.

The gesture-recognition model 621 selects a gating-sensitivity atoperation 1406, based on the identity of the subscriber. Thegesture-recognition model 621 determines, based on thegating-sensitivity associated with the identity of the subscriber,whether the context satisfies the requirements for radar-basedgesture-recognition with the radar system 104.

At 1710, an operation is performed in response to the model determininga radar-based (e.g., touch-independent) gesture. An output from thegesture-recognition model 621 can indicate a gesture recognized from theradar data and output an indication of the gesture to a sub scriber.

The UE 102 may provide user-interface feedback of a gating state of theradar system 104. The UE 102 can output an audible or visual indicationto a user, such as audible or visual alert (e.g., “you are moving thedevice too much and the radar cannot sense your gesture”), controlling alighting element of the UE 102, providing haptic feedback, or providingsome other user-interface feedback. The UE 102 may output an indicationof the gating state as being “gating” or “not gating” to indicatewhether the UE 102 is gating the output from the gesture-recognitionmodel 621 or not. The indication of the gating state can indicate areason for gating (e.g., providing an indication of a contextual orenvironmental characteristic that makes gating necessary). Theindication of the gating state can indicate a level of gating (e.g., seeFIG. 18 for example levels of gating including soft-gating, hard-gating,and no gating).

The UE 102 can vary a user interface and provide user-interface feedbackin other ways. For example, if the UE 102 is being used, a display ison, and the UE 102 is operating in a high-power state, user-interfacefeedback being output from the UE 102 may depend only on sensor datafrom motion sensors or other non-radar sensors. If the display is off orthe UE 102 is in a lower-power state, it may be prohibitive to operatethe motion sensor or other non-radar based sensor in an always-enabledstate. The UE 102 may refrain from monitoring the motion sensors orother non-radar sensors except in contexts that satisfy requirements fortouch-independent gesture-recognition. In this way, the user-interfacefeedback is conditioned on whether the gesture-recognition model 621 candetermine a radar-based gesture.

For example, the UE 102 can provide a “gesture” user-interface feedbackelement when soft or hard-gating the radar system 104 and/or when gatingceases and radar-based gesture-recognition resumes. A gestureuser-interface feedback element is a user-perceivable element, such as avisual element that appears on an active area of a display. A gesturefeedback element can also be (or include) a light element that is not onthe display, a haptic element (e.g., a vibration element), and/or anaudio element (e.g., a user-perceivable sound), may be presented at oralong an edge of a display, and may have any of a variety of shapes,sizes, colors, and other visual parameters or properties. Examples ofthe other visual parameters or properties include luminosity, color,contrast, shape, saturation, or opaqueness.

Gesture-Recognition Gating

FIG. 18 depicts an example method 1800 for radar-basedgesture-recognition with context-sensitive gating and othercontext-sensitive controls. The method 1800 is shown as a set of blocksthat specify operations performed but are not necessarily limited to theorder or combinations shown for performing the operations by therespective blocks. Further, any of one or more of the operations may berepeated, combined, reorganized, or linked to provide a wide array ofadditional and/or alternate methods, e.g., methods 1000, 1200, 1400, and1700. In portions of the following discussion, reference may be made tothe example operating environment 100 of FIG. 1 or to entities orprocesses as detailed in other figures, reference to which is made forexample only. The techniques are not limited to performance by oneentity or multiple entities operating on one device.

There are two common scenarios for performing the operations 1802, 1804,and 1806 to employ gating of a radar-based detection system. Onescenario is when the radar system 104 is covered or occluded by anobject. The radar system 104 may be occluded or covered when the UE 102is lying face down on a surface, or is in a pocket, a purse, a bag, orother enclosure. The other scenario is when the UE 102 is experiencingsignificant motion. For example, if the user 120 carrying the UE 102 intheir hand goes for a run, the UE 102 should not incorrectly interprettouch-independent gestures with the radar system 104.

At 1802, whether to hard-gate radar-based gesture-recognition by theradar system 104 is determined. A context is determined, indicatingwhether the radar system is occluded (e.g., from a user) by an object.The UE 102 selects from multiple levels of gating based on the context,including hard-gating and soft-gating, and then gates thegesture-recognition model 621 accordingly.

As used herein, the term “soft-gating” references an operation thatblocks indications of radar-based gestures, from being output by theradar system 104 and to subscribers. Unlike hard-gating where the radarsystem 104 operates in a low-power mode or intermediate-power mode,soft-gating occurs without regard to power levels of the radar system104. Soft-gating can occur by disabling the output from thegesture-recognition model 621, and in other cases, soft-gating happensfrom disabling the input to the gesture-recognition model 621. Thegesture-recognition model 621 may continue to recognize radar-basedgestures during soft-gating. However, during soft-gating, the radarsystem 104 does not share the recognized gestures with subscribers(e.g., applications, threads, activities, user interface objects). Thesubscribers do not receive indications of recognized gestures for theiruse in performing higher-level functions.

During soft-gating, the gesture-recognition model 621 can be shieldedfrom radar data collected by the radar system 104, and in other timesduring soft-gating, a radar-based gesture determination is made by thegesture-recognition model 621 anyway but used internally by the radarsystem 104, for some other purpose, e.g., a system service or hiddenfunction. During soft-gating, the UE 102 may still perform lower-levelsupport functions based on indications of gestures recognized by thegesture-recognition model 621, however, the support functions may betransparent to subscribers and users of the UE 102. Support functionsinclude learning, understanding, and acting upon gestures, even during agating-context, to minimize potential latency from soft-gating thegesture-recognition model 621 in the future.

Contrast soft-gating with the term “hard-gating” which, as used herein,refers to an operation that triggers the radar system 104 to function ina state during which the radar system 104 does not recognize gesturesfrom radar data. The gesture-recognition model 621 is disabled duringhard-gating. During hard-gating, the radar system 104 can be used forother tasks besides gesture-recognition. Depending on whether the UE 102needs the radar system 104 for any other capability, other parts of theradar system 104 may or may not be disabled during a hard-gating contextas well. Thus, while the radar system 104 may continue to perform otherfunctions unrelated to radar-based gesture-recognition, such as obstacleavoidance, the gesture-recognition model 621 of the radar system 104does not output indications of recognized gestures when beinghard-gated, thereby offering some power consumption savings oversoft-gating or not at all gating the radar system 104. In addition toproviding savings in power consumption, hard-gating is particularlyuseful for improving a user-experience by preventing subscribers of theradar system 104 from performing higher-level functions in response tofalse or unintended input.

The UE 102 may have improved latency recovering from an inactivegesture-recognition state if the UE 102 is soft-gating the radar system104. With hard-gating, increased latency in recovering from an inactivegesture-recognition state (e.g., where the gesture-recognition featureof the radar system 104 may be powered-off) is offset by power savedfrom not executing complex gesture-recognition functions or disruptinghigh-level functions from false inputs. That is, while hard-gating theradar system 104 prevents unnecessary power consumption interpretinggestures from radar data during contexts when the UE 102 is unlikely toreceive input from a user, the UE 102 may be slower than if the radarsystem 104 were soft-gated to transition back to a normal operating modewhen gating is no longer necessary.

At 1804, the UE 102 performs hard-gating by setting the radar system 104to an intermediate-power or low-power mode for outputting no data orother data that is unusable by the gesture-recognition model 621 fordetermining the touch-independent gesture. When the radar system 104 isoccluded, an output from the radar system 104 is hard-gated at 1804 bydisabling the gesture-recognition model 621.

At 1802, e.g., when the radar system 104 is not occluded, the outputfrom the radar system is soft-gated at 1806. The UE 102 soft-gates theradar system 104 by refraining from inputting the radar data obtained bythe radar system 104 to the gesture-recognition model 621.Alternatively, the UE 102 soft-gates the radar system 104 by preventingthe gesture-recognition model 621 from outputting indications ofrecognized gestures.

The radar system 104 can transition between no-gating, soft-gating, andhard-gating depending on the context, during subsequent execution of theoperations 1700 and 1800. For example, after soft-gating or hard-gatingthe radar system 104, the method of FIG. 18 returns to “A” and the startof operations 1700 of FIG. 17 . If after soft-gating the radar system,at 1706, 1802 it is determined that the context indicates the radarsystem 104 is occluded by proximity to the object, the radar system 104is hard-gated.

Gating Sensitivity

FIG. 19 illustrates a decision tree that implements the methods of FIGS.17 and 18 . Portions of the scheme 1900 may be performed by theprocessor 608, the computer processors 402, or other hardware circuitry.The scheme 1900 can be customized to support different types ofelectronic devices and radar-based applications.

Sensor data reception 1902 occurs as the UE 102 performs the operation1702. From the operation 1702, the UE 102 uses the sensor data toexecute motion detection 1904 and proximity-detection algorithms on thesensor data to develop a context during operations 1704, includingdetermining whether the radar system 104 is occluded, or increased rateof motion is determined.

The sensitivity of the motion detection 1904 and the proximity detection1906 can be selected to balance motion-gating behavior while being ableto reject device motion. Two common scenarios illustrate a need forsensitivity adjustments. A first scenario is if the user 120 is slowlywalking with the UE 102 carried at the user's side and swinging pasttheir body. Without persistent gating, the first scenario could causesignificant false triggering. A second scenario is if the user 120 liftsthe UE 102 to act. When lifting the UE 102 (from the side, from a table,from a pocket, etc.) to interact, the subsequent detection of naturalbody motions should invoke a rapid gating response from the UE 102. Alower gating sensitivity is needed for the response to be satisfyinglyswift and not cause any delayed interference for the user. The gatingdecision 1908 is made during operation 1706, which leads to one of threegating modes: off 1910, soft-gate 1912, or hard-gate 1914. For furtherdetails about varying gating sensitivity, see the section “MovementBased Gating” and description of FIG. 21 below.

Gating State Machine

FIG. 20 illustrates a state diagram for a state machine 2000 thatimplements the methods of the FIGS. 17 and 18 . The state machine 2000is a gating state machine and may execute as part of the radar system104.

The state machine 2000 includes multiple states 2002, 2004, and 2006,each linked by respective context-sensitive transition functions 2008-1through 2008-6 (collectively “functions 2008”). Each of the functions2008 receives at least a portion of the sensor data, or a derivationthereof, as variable inputs. For ease of description, the state machine2000 includes only three states: 2002, 2004, and 2006. In otherexamples, more than three states are used by the state machine 2000. Thestate machine 2000 transitions between the states 2004, 2006, and basedon the functions 2008.

The state machine 2000 includes a no-gating state 2002 in whichradar-based gesture-recognition with the radar system 104 is enabled. Asoft-gating state 2004 is where radar-based gesture-recognition with theradar system 104 is enabled but results of the radar-basedgesture-recognition are withheld from applications and other subscribersexecuting at the UE 102. For a hard-gating state 2006, the radar-basedgesture-recognition functionality of the radar system 104 is disabled,although other functions of the radar system 104 may remain enabled(e.g., the radar system 104 can execute a collision-avoidance functionduring hard-gating when gesture-recognition is disabled).

Each of the functions 2008 computes a respective contextual scoreindicating compatibility between a current context and each of thestates 2002, 2004, and 2006. For example, the function 2008-3 computes acontextual score indicating whether, based on sensor data that defines acurrent context, the state machine 2000 should transition to thesoft-gate state 2004. The function 2008-6 computes a contextual scoreindicating whether, based on the sensor data that defines the currentcontext, the state machine 2000 should transition to the hard-gate state2006. The state machine 2000 transitions from the no-gate state 2000 toeither the soft-gate state 2004 or the hard-gate state 2006, based onwhich of the two states 2004 or 2006 has a contextual score thatsatisfies a transition threshold. If each of the functions 2008-3 and2008-6 compute a contextual score that satisfies a transition thresholdto transition the state machine 2000 to a next state, the state machine2000 may transition to the next state with the highest contextual score.

When in the no-gating state 2002, the state machine 2000 of the radarsystem 104 receives sensor data from the sensors 108. The functions2008-3 and 2008-6 take the sensor data as inputs and compute contextualscores indicating whether the sensor data satisfies the requirements forentering the soft-gating state 2004 or the hard-gating state 2006,respectively. The function 2008-3 corresponds to a “No” outcome from theoperation 1802 of FIG. 18 . The function 2008-6 corresponds to a “Yes”outcome from the operation 1802 of FIG. 18 . If neither contextual scoreout of the functions 2008-3 and 2008-6 satisfies a respective transitionthreshold, the state machine 2000 remains in the no-gating state 2002.

In a context where the sensor data indicates the user 120 is holding theUE 102 and viewing the UE 102, the state machine 2000 keeps the radarsystem 104 in a gesture-recognition mode operating in the no-gatingstate 2002. If the user 120 looks away from the UE 102 to talk toanother person without dropping the UE 102 or maintaining the UE 102substantially steady, the function 2008-3 may compute a contextual scorethat exceeds a respective transition threshold for transitioning to thesoft-gating state 2004. The UE 102 may want to remain ready to resumedetecting radar-based user inputs, so in a situation such as this wherethe user temporarily disengages from the UE 102, the UE 102 can quicklyreturn to the no-gating state 2002 if the user 120 looks back to the UE102; soft-gating thereby enhances the user experience with the UE 102.The state machine 2000 transitions to the soft-gating state 2004 andcontinues to enable radar-based gesture recognition with the radarsystem 104, however the state machine 2000 prevents the radar system 104from outputting results of the gesture-recognitions to applicationsexecuting at the UE 102.

Starting from the no-gating state 2002 again, in a slightly differentcontext where the sensor data indicates the user 120 looking away fromthe UE 102 to talk to another person while also dropping the UE 102 tothe user's 120 side, or otherwise not maintaining the UE 102substantially steady. The function 2008-6 may compute a contextual scorethat exceeds a respective transition threshold for transitioning to thehard-gating state 2006. The radar system 104 can continue to performother radar operations for the UE 102, however the radar-basedgesture-recognition function of the radar system 104 is disabled in thehard-gating state. Hard-gating thereby promotes power savings, placingthe radar state 104 in a state where gesture-recognition is disabled,when gesture-recognition is not likely to be needed.

After transitioning to the soft-gating state 2004, updated sensor datais received from the sensors 108 and the radar system 104. The statemachine 2000 computes a respective contextual score using the functions2008-1 and 2008-4. The function 2008-1 corresponds to a “Yes” outcomefrom the operation 1706 of FIG. 17 . The function 2008-4 corresponds toa “Yes” outcome from the operation 1802 of FIG. 18 . If the contextualscore of the function 2008-1 exceeds a transition threshold fortransitioning to the no-gating state 2002, the state machine 2000transitions to the no-gating state 2002. If the contextual score of thefunction 2008-4 exceeds a transition threshold for transitioning to thehard-gating state 2006, the state machine 2000 transitions to thehard-gating state 2006. If both contextual scores of the functions2008-1 and 2008-1 exceed their respective transition thresholds, thestate machine 2000 may transition to the state 2002 or 2006, which isassociated with a higher-contextual score than the contextual score ofthe other function. Assume the contextual score of the function 2008-4exceeds the contextual score of the function 2008-1 and the transitionthreshold associated with transitioning from the soft-gating state 2004to the hard-gating state 2006.

After transitioning to the hard-gating state 2006, updated sensor datais received from the sensors 108 and the radar system 104. The statemachine 2000 computes a respective contextual score using the functions2008-2 and 2008-5. The function 2008-2 corresponds to a “No” outcomefrom the operation 1802 of FIG. 18 . The function 2008-5 corresponds toa “Yes” outcome from the operation 1706 of FIG. 17 . If the contextualscore of the function 2008-5 exceeds a transition threshold fortransitioning to the no-gating state 2002, the state machine 2000transitions to the no-gating state 2002. If the contextual score of thefunction 2008-2 exceeds a transition threshold for transitioning to thesoft-gating state 2004, the state machine 2000 transitions to thesoft-gating state 2004. If both contextual scores of the functions2008-2 and 2008-5 exceed their respective transition thresholds, thestate machine 2000 may transition to the state 2002 or 2004 which isassociated with a higher-contextual score than the contextual score ofthe other function.

The state machine 2000 can be machine-learned or driven based oninferences made by a machine-learned model. The machine-learned model istrained to predict a suitable gating state for the radar system 104,based on sensor data or other input that defines a current context. Forexample, the functions 2008 can be machine-learned rules or applicationsor the machine-learned model to a current context to compute acontextual score. Said differently, each of the functions 2008 can be amachine-learned model, or instance of a machine-learned model, trainedto predict the next radar state or a contextual score equating thecurrent context to a next radar state.

Other Context-Sensitive Controls

As described above in great detail, the radar system 104 relies oncontext of the UE 102 and an awareness of the user's 120 location andposition to gate the radar system 104 or not gate the radar system 104.These same techniques that apply to context-sensitive gating can applyto other context-sensitive controls that rely on the radar system 104and radar functionality.

The UE 102 can also use the radar system 104 and other of the sensors108 to predict the user's 120 intent to engage. The context of the UE102 can be relative to the user 120, indicating the distance from the UE102 to the user 120, indicating whether the user 120 is moving toward oraway from the UE 102, indicating whether the user 120 is reaching forthe UE 102, and the posture or orientation of the user 120 related tothe UE 102.

The radar system 104 reconfigures how gesture-recognition, proximitydetection, and other radar functions are performed, to adapt each radarfunction to best suit a current context. For example, the distances andsensitivities programmed into the functions 2008 for transitioningbetween the different states 2002, 2004, and 2006 of the state machine2000 when the user 120 and the UE 102 are in a medium-sized room may notbe appropriate in some contexts. If the user 120 and UE 102 are in asmaller room, an automobile, or even in the medium-sized room with adifferent quantity of people than originally predicted, the functions2008 for transitioning between the different states 2002, 2004, and 2006change or adapt to fit the new context. Said differently, the statemachine 2000 can include transition functions, such as the functions2008, which dynamically change criteria based on changes in context. Theresults of the functions 2008 may likewise change accordingly. Based oninput from an available signal, sensor, or other data, the state machine2000 can adjust parameters to the functions 2008 and thereby adjust thefunctionality of the UE 102. As mentioned above, the functions 2008 maybe machine-learned models or portions of a machine-learned model,trained to predict a confidence or score that a particular state issuited for a current context. The following are some non-limitingexamples of how the radar system 104 dynamically adapts radar functionsto best suit a current context.

The state machine 2000 can pause (e.g., soft-gate, hard-gate) the radarsystem 104 (or put the radar system 104 in a sleep mode) based oninertial data generated by an IMU of the sensors 108. Inertial dataindicating the UE 102 is moving in a way that may reduce the accuracy orefficiency of the radar system's 104 ability to perform other radarfunctions, not only radar-based gesture-recognition. The inertial datafrom the IMU can include X, Y, and Z-axis movement information. Thestate machine 2000 combines the three movements into a floating-pointvalue that the state machine 2000 inputs into the functions 20008 fortransitioning between the states 2002, 2004, and 2006.

The state machine 2000 controls the radar system 104 (or put the radarsystem 104 in a sleep mode) based on other non-IMU sensor data generatedby the sensors 108 as-well, or any other useful information generated byany other data source. For example, the UE 102 may include a calendarapplication, a clock application, location services, proximity services,communication services, financial services, or any other contextual datasource. A subscriber application executing at the UE 102 may provide theradar system 104 with contextual information just as the subscriberapplication may receive indications of gesture-inputs recognized by theradar system 104.

All of these potential sources of information can feed the state machine2000 and the functions 2008 to determine whether the radar system 104should be paused or gated. Additionally, the system can know whatapplications are running, which can further refine the contextualawareness of the UE 102 and help the UE 102 make a decision regardingthe pause mode.

Contextual awareness by the radar system 104 further enables the UE 102to change a number of available radar states or modes, depending oncontext. For example, in an automobile context, the radar system 104need only be in a no-gating or soft-gating mode, because maximumresponsiveness without regard to power consumption is a desirablecharacteristic of the UE 102 when in an automobile mode (if on theautomobile's power). Only two states are necessary, for example, becausethe radar system 104 assumes the user 120 is only a few feet away(confined to the automobile) so hard-gating when the user is not presentor to save power when not likely to be interacting with the UE 102 isnot necessary.

The contextual awareness by the radar system 104 relies on dynamicfunctions 2008 or even machine-learned models to adjust the triggerparameters between gating states, and other radar modes of the radarsystem 104, such as the size of an awareness zone or a recognition zone,the sensitivity to changes in distance or speed of reaches or othergestures, etc. Other functionality of the radar system 104 can becontext-based. Consider a user alone in an automobile versus a user on asubway or in a crowded meeting room. The radar system 104 can determineradar-based gestures using different sensitivities, feedback, andfeatures, because certain settings such as these may be more effectivein different contexts.

Contextual awareness for controlling the radar system 104 can be usefulin other ways. For example, in response to detecting the UE 102 in astowed context, e.g., on a bicycle, the radar system 104 mayautomatically configure itself for crash avoidance radar mode anddisable gesture-recognition.

The radar system 104 may be more effective when stable. If the sensordata from the sensors 108 indicates the UE 102 is shaking or vibratingat too high a magnitude or frequency of shake or vibration, the radarsystem 104 automatically disables radar-based gesture-recognition andother radar functionality. This saves a lot of unnecessary computing andmeasurement cycles because when the UE 102 is not stable and shaking,the radar system 104 does not likely provide useful results.

A contextual information source to the UE 102 can be remote to the UE102, for example, a sensor or input component of a computerized watchthat is paired with the UE 102 can be a further source of sensorinformation that supplements the sensor data collected from the sensors108. In this case, the radar system 104 may gate or otherwise controlthe radar functionality based on sensor data from a communicativelycoupled watch. The sensor data could include heart-rate information.When the user's heart-rate exceeds a particular threshold for indicatingexercise or intense physical movement, the radar system 104 may disablethe radar-based gesture recognition or other feature of the radar system104 as the user is not likely to be gesturing at the UE 102 whenexercising.

An ambient light sensor from the sensors 108 captures sensor dataindicating when the context of the UE 102 is in a low-lit area. In sucha context, the radar system 104 operates under an assumption that theuser 120 will have a hard time interacting with the UE 102 and thereforethe radar system 104 makes its interface more forgiving to sloppyinputs.

A proximity sensor from the sensors 108, e.g., an optical proximitysensor, can trigger the radar system 104 to switch-off or enter a stateduring which gesture-recognition is disabled, when the radar system 104is occluded. Wireless signals, power connections, network connections,and other connections to the UE 102 can provide additional contextualinformation for controlling the radar system 104. In response todetecting a charging cable, docking station, or wireless charging systempowering the UE 102, the radar system 104 refrains from entering thehard-gating state 2006 as the UE 102 does not need to deal with powerconsumption when charging and the user 120 would more likely want afaster response rate from the radar system 104. In a related example,when connected to a wireless charging system, the radar system 104 maydisable much of its capability to avoid interfering with wirelesschargers. The radar system 104 may operate in an inactive mode to avoidinterfering with communications and other signals transmitted orreceived by the UE 102.

The radar system 104 can be operatively coupled to one or more of thesensors 108 and trigger in response to interrupts or informationreceived directly from the sensors 108. For example, anear-field-communication unit or NFC sensor can trigger the radar system104 to enter a no-gating mode when the NFC is processing a payment orother authentication gesture.

The radar system 104 can switch-on or switch-off in coordination withother input components. For example, the user 120 may provide input to atouchscreen of the UE 102, and while detecting an input at thetouchscreen, the radar system 104 may disable gesture-recognition. Inother cases, the radar system 104 enhances the touchscreen functionalityby remaining switched-on and sending information about recognizedgestures to an input decoder that processes touchscreen data and radardata simultaneously to infer user intent. In this way, the radar system104 and a touchscreen can recognize typing at soft-keyboard or otherinput to a GUI, even if the user 120 wears gloves while providing touchinput, which can interfere with some presence-sensitive screens.

The radar system 104 can control radar functions based on othercontextual information, including temperature, humidity, pressure, etc.The radar system 104 may use certain settings to account for performancevariations that can occur for variations in meteorological conditions.Using voice or sound information, the radar system 104 can control theradar functions, activating or deactivating features based on voicecommands.

Movement-Based Gating

FIG. 21 illustrates a block diagram 2100 for implementing movement-basedgating of radar-based gesture-recognition. Portions of the block diagram2100 may be performed by the processor 608, the computer processors 402,or other hardware circuitry. The block diagram 2100 can be customized tosupport different types of electronic devices and radar-basedapplications.

The block diagram 2100 includes a low-pass filter (e.g., a fourth-orderButterworth filter) and digital blocker 2102. The inertial sensor data(e.g., from the IMU 408) is filtered for all three axes (e.g., X, Y, andZ). Filtering may occur at a different rate than the rate of output fromthe IMU 408. For example, the IMU 408 outputs accelerations at fiftyhertz, yet the low-pass filter and digital blocker 2102 filters theaccelerations at twelve hertz. The low-pass filter and digital blocker2102 output acceleration measurements to a significant-motion detector2120 and a phone-orientation detector 2122.

The significant-motion detector 2120 determines an absolute value forthe inertial sensor data for each of the axes (e.g., X, Y, and Z) usingan absolute value component 2104. An envelope component 2106 buffers thefiltered and normalized acceleration measurements, for evaluation by anadjustable threshold component 2108 that detects significant motionrelative to historically observed motion over time (hysteresis).

The phone-orientation detector 2122 determines the orientation of the UE102. The sensitivity of the gating function applied to the radar system104 varies according to changes in the orientation of the UE 102. Thesensitivity of the gating function is also based an amount of movementof the UE 102 given the orientation.

A moving-average component 112 calculates average motion per each of theaxes (e.g., X, Y, and Z). A “dot product with reference vector” (DPRV)component 2114 determines values from the average motions that arerepresentative of orientation and movement of the UE 102. Theorientation and movement values are received by an adjustable thresholdcomponent 2116. If the orientation and movement values received by theadjustable threshold component 2116 indicate a particular orientation ormovement, the adjustable threshold component 2116 sets the adjustablethreshold component 2108 to a high sensitivity or a low sensitivity. Forexample, if the phone-orientation detector 2122 determines that the UE102 is likely in landscape or a portrait-down orientation, the thresholdcomponent 2108 may be set by the threshold component 2116 to a highsensitivity so the radar system 104 is gated more of often to preventsmall movements or changes in orientation from being used ingesture-recognition. If the phone-orientation detector 2122 determinesthat the UE 102 is likely in a different orientation, the thresholdcomponent 2108 may be set by the threshold component 2116 to alower-sensitivity so the radar system 104 is gated less often and toenable even small movements or changes in orientation to be used ingesture-recognition. In other words, the output from the adjustablethreshold 2116 may act like a switch that directs the adjustablethreshold of 2108 to have a high or low sensitivity depending onmovement and orientation.

The output of the block diagram 2100 is a gating decision determinedfrom a combination of the significant-motion detector 2120 and thephone-orientation detector 2122. A logical OR component 2110 outputs aparticular value depending on whether to gate the radar system 104 ornot. Depending on the gating decision from the logical OR component2110, the radar system 104 may soft-gate or not gate an output.

The sensitivity of the significant-motion detector 2120 is tied to thephone-orientation detector 2122 to balance the motion-gating behavior ofthe radar system 104 while also being able to reject device motion.Slowly walking with the UE 102 carried at the user 120's side andswinging past their body can cause significant false triggering withoutadjusting gating sensitivity of the radar system 104 as described above.If the user 120 lifts the UE 102 to act (from the side, from a table,from a pocket, etc.) the subsequent detection of natural body motionsshould invoke a rapid gating response from the UE 102 and as such, alower gating sensitivity is needed for the response from the radarsystem 104 to be satisfyingly swift and not cause any delayed annoyancefor the user 120.

EXAMPLES

In the following paragraphs, examples are provided.

Example 1. A method comprising: receiving, from a plurality of sensorsof a user equipment, sensor data; determining, based on the sensor data,a context of the user equipment; determining whether the contextsatisfies requirements for radar-based gesture-recognition; andresponsive to determining that the context does not satisfy therequirements for radar-based gesture-recognition, gating the radarsystem to prevent the radar system from outputting indications ofradar-based gestures to application subscribers of the user equipment.

Example 2. The method of example 1, wherein gating the radar systemcomprises hard-gating the radar system by triggering the radar system tofunction in a state during which the radar system does not recognizegestures from radar data.

Example 3. The method of example 2, wherein hard-gating the radar systemis further responsive to determining that the context indicates that theradar system is occluded by an object.

Example 4. The method of example 1, wherein gating the radar systemcomprises soft-gating the radar system by triggering the radar system tofunction in a state during which the radar system does not outputindications of the radar-based gestures.

Example 5. The method of example 4, wherein soft-gating the radar systemis further responsive to determining that the context indicates that theradar system is not occluded by the object.

Example 6. The method of example 4, wherein soft-gating the radar systemto prevent the radar system from outputting the indications of theradar-based gestures to the application subscribers of the userequipment does not prohibit the radar system from recognizing theradar-based gestures from radar data.

Example 7. The method of example 4, further comprising: aftersoft-gating the radar system, determining that the context indicates theradar system is occluded the object; and responsive to determining thatthe context indicates the radar system is occluded by the object,hard-gating the radar system by triggering the radar system to functionin a state during which the radar system does not recognize gesturesfrom radar data.

Example 8. The method of example 1, wherein the context is a firstcontext and the sensor data is first sensor data, the method furthercomprising: receiving, from the plurality of sensors, second sensordata; determining, based on the second sensor data, a second context ofthe user equipment; determining whether the second context satisfies therequirements for radar-based gesture-recognition; responsive todetermining that the second context satisfies the requirements forradar-based gesture-recognition, inputting radar data obtained by theradar system to a model that determines radar-based gestures from theinputted radar data; and performing an operation in response to themodel determining a radar-based gesture, the operation associated withthe determined radar-based gesture.

Example 9. The method of example 8, wherein inputting the radar dataobtained by the radar system to the model for radar-basedgesture-recognition comprises refraining from gating the radar systemand setting the radar system to an active state for radar-basedgesture-recognition.

Example 10. The method of any of examples 1 through 9, wherein the radarsystem is configured as a proximity sensor for generating at least aportion of the sensor data.

Example 11. The method of any of examples 1 through 10, whereindetermining whether the context satisfies the requirements forradar-based gesture-recognition with the radar system comprisesdetermining whether the context indicates a user is holding the userequipment or whether the context indicates the user is walking.

Example 12. The method of example 11, further comprising: determiningthe context does not satisfy the requirements for radar-basedgesture-recognition in response to determining the user is not holdingthe user equipment and the user is walking; or in response todetermining the radar system is occluded by an object.

Example 13. The method of example 11, further comprising: determining,based on whether the sensor data indicates a particular movement,whether the user is holding the user equipment, how the user is holdingthe user equipment, or whether the user is walking.

Example 14. The method of any of examples 1 through 13, furthercomprising: determining an identity of an application-subscriber of theradar-based gesture-recognition; selecting, based on the identity of thesubscriber, a gating-sensitivity for determining whether the contextsatisfies the requirements for radar-based gesture-recognition, whereindetermining whether the context satisfies the requirements forradar-based gesture-recognition with the radar system is based on thegating-sensitivity associated with the identity of the subscriber.

Example 15. The method of example 14, wherein the gating-sensitivity isspecific to a type of radar-based gesture preselected by one of theapplication subscribers.

Example 16. The method of any of examples 1 through 15, furthercomprising: changing a state of the user equipment in response to themodel determining a radar-based gesture, the state of the user equipmentincluding an access-state, a power state, or an information state.

Example 17. The method of any of examples 1 through 16, whereindetermining whether the context satisfies requirements for radar-basedgesture-recognition comprises executing a state machine comprisingmultiple states linked by respective context-sensitive transitionfunctions that receive at least a portion of the sensor data as variableinputs.

Example 18. The method of example 17, wherein the state machinecomprises a no-gating state in which radar-based gesture-recognitionwith the radar system is enabled, a soft-gating state in which theradar-based gesture-recognition with the radar system is enabled butresults of the radar-based gesture-recognition are withheld fromapplications and other subscribers executing at the user equipment, anda hard-gating state in which the radar-based gesture-recognition isdisabled.

Example 19. The method of any of examples 1 through 18, wherein theplurality of sensors comprise an inertial measurement unit.

Example 20. The method of any of examples 1 through 19, wherein theplurality of sensors exclude camera sensors.

Example 21. The method of any of examples 1 through 20, wherein theplurality of sensors include a proximity sensor, an ambient lightsensor, a microphone, or a barometer.

Example 22. The method of example 21, wherein the proximity sensor is anoptical proximity sensor.

Example 23. An apparatus comprising: a radar system that detectsradar-based gestures on behalf of application subscribers; an inertialmeasurement unit that receives inertial sensor data; and a state machinethat transitions between multiple states for controlling the radarsystem based on the inertial sensor data and context-sensitivetransition functions, the state machine including: a no-gating state inwhich the state machine enables the radar system to output indicationsof the radar-based gestures to the application subscribers; asoft-gating state in which the state machine prevents the radar systemfrom outputting the indications of the radar-based gestures to theapplication subscribers; and a hard-gating state in which the statemachine prevents the radar system from detecting the radar-basedgestures.

Example 24. The apparatus of example 23, wherein at least one of thecontext-sensitive transition functions receives at least a portion ofthe inertial sensor data as variable input.

Example 25. The apparatus of example 23 or 24, wherein the state machineenables the radar system to detect the radar-based gestures whileprevented from outputting the indications of the radar-based gestures tothe application subscribers.

Example 26. The apparatus of any of examples 23-25, wherein: a firstfunction from the context-sensitive transition functions outputs a firstcontextual score indicating whether, based on the inertial sensor data,to transition from the no-gating state to the soft-gating state; and thestate machine transitions from the no-gating state to the soft-gatingstate in response to determining that the first contextual scoresatisfies a first threshold.

Example 27. The apparatus of example 26, wherein: a second function fromthe context-sensitive transition functions outputs a second contextualscore indicating whether, based on the inertial sensor data, totransition from the no-gating state to the hard-gating state; and thestate machine transitions from the no-gating state to the hard-gatingstate in response to determining that the second contextual scoresatisfies a second threshold.

Example 28. The apparatus of example 27, wherein the state machinetransitions from the no-gating state to the hard-gating state inresponse to determining that the second contextual score satisfies thesecond threshold and the second contextual score exceeds the firstcontextual score.

Example 29. The apparatus of example 28, wherein the state machinetransitions from the no-gating state to the soft-gating state inresponse to determining that the first contextual score satisfies thefirst threshold and the fist contextual score exceeds the secondcontextual score.

Example 30. The apparatus of any of the examples 23-29, wherein theapparatus consumes a different amount of electrical power when the statemachine operates in the soft-gating state than when the state machineoperates in the hard-gating state.

Example 31. The apparatus of any of the examples 23-30, wherein theapparatus consumes more electrical power from the detecting theradar-based gestures when the state machine operates in the soft-gatingstate than when the state machine operates in the hard-gating state.

Example 32. The apparatus of any of the examples 23-31, wherein at leastone of the context-sensitive transition functions outputs a contextualscore based on a comparison between a position, orientation, or movementinferred from the inertial sensor data and a position, orientation, ormovement threshold.

Example 33. The apparatus of any of the examples 23-32, wherein a firstfunction from the context-sensitive transition functions outputs acontextual score based on whether the radar system is occluded by anobject.

Example 34. The apparatus of example 33, wherein the state machinetransitions from the no-gating state to the soft-gating state inresponse to the contextual score indicating that the apparatus is notoccluded by the object.

Example 35. The apparatus of any of the examples 33 or 34, wherein thestate machine transitions from the no-gating state to the hard-gatingstate in response to the contextual score indicating that the apparatusis occluded by the object.

Example 36. The apparatus of any of the claims 23-35, wherein at leastone of the context-sensitive transition functions receives at least aportion of the inertial sensor data as variable input after filteringthe inertial sensor data through a low-pass filter.

Example 37. The apparatus of any of the claims 23-36, wherein at leastone of the context-sensitive transition functions receives an indicationof an orientation of the apparatus as variable input and the statemachine transitions between multiple states for controlling the radarsystem based on the inertial sensor data, the context-sensitivetransition functions, and the orientation.

Example 38. The apparatus of any of the claims 23-37, wherein at leastone of the context-sensitive transition functions receives an indicationof proximity from the apparatus to an object as variable input and thestate machine transitions between multiple states for controlling theradar system based on the inertial sensor data, the context-sensitivetransition functions, and the proximity.

Example 39. A user equipment comprising the apparatus of any one of theexamples 23 through 38 and a processor configured to execute the statemachine.

Example 40. A system comprising means to execute the state machine ofthe apparatus of any of the examples 23 through 38.

CONCLUSION

Although implementations of techniques for, and apparatuses enabling,radar-based gesture-recognition with context-sensitive gating and othercontext-sensitive controls have been described in language specific tofeatures and/or methods, it is to be understood that the subject of theappended claims is not necessarily limited to the specific features ormethods described. Rather, the specific features and methods aredisclosed as example implementations enabling radar-basedgesture-recognition with context-sensitive gating and othercontext-sensitive controls.

What is claimed is:
 1. An apparatus comprising: a radar system thatdetects radar-based gestures on behalf of application subscribers; aninertial measurement unit that produces inertial data; and a statemachine configured to transition between multiple states for controllingthe radar system based on the inertial data and one or morecontext-sensitive transition functions, the multiple states including: ano-gating state in which the state machine enables the radar system tooutput indications of the radar-based gestures to the applicationsubscribers; and a soft-gating state in which the radar system isprevented from outputting the indications of the radar-based gestures toone or more of the application subscribers by shielding agesture-recognition model from radar data collected by the radar system.2. The apparatus of claim 1, wherein the state machine further includesa hard-gating state in which the state machine prevents the radar systemfrom detecting the radar-based gestures.
 3. The apparatus of claim 2,wherein the hard-gating state further includes operating the radarsystem in a low-resolution mode that generates low-resolution radar datathat is effective to disable the gesture-recognition model.
 4. Theapparatus of claim 2, wherein the soft-gating state requires the radarsystem to utilize more power than the hard-gating state.
 5. Theapparatus of claim 1, further comprising: one or more sensors thatreceive sensor data; and a machine learned activity classifier that usesmachine learning techniques to recognize patterns or signatures in thesensor data to determine a device context, wherein the transitionsbetween the multiple states for controlling the radar system are basedon the inertial data and the determined device context.
 6. The apparatusof claim 5, wherein the one or more sensors include the inertialmeasurement unit and the sensor data includes the inertial data.
 7. Theapparatus of claim 5, wherein the one or more sensors include at leastone of: an ambient light sensor, a barometer, a location sensor, anoptical sensor, an infrared sensor, an accelerometer, a gyroscope, amagnetometer, a compass, or a temperature sensor.
 8. The apparatus ofclaim 5, wherein the device context is based on at least one of: adetermination that the apparatus is being held by a user, adetermination that the apparatus is moving, a determination that theapparatus is stowed, a determination that the apparatus is oriented in aparticular way, a determination that the apparatus is in a particularlocation, or a determination that at least one of the one or moresensors is occluded.
 9. The apparatus of claim 5, wherein the devicecontext includes at least one of: a walking context, a cycling context,a driving context, a riding context, a seated context, a stationarycontext, an unused context, or a stowed context.
 10. The apparatus ofclaim 1, wherein the soft-gating state prevents the radar system fromoutputting the indications of the radar-based gestures to a first set ofthe application subscribers and a second set of the applicationsubscribers.
 11. The apparatus of claim 10, wherein the first set of theapplication subscribers and the second set of the applicationsubscribers are determined based on one or more device contexts.
 12. Theapparatus of claim 1, wherein the soft-gating state enables indicationsof the radar-based gestures to be used for a lower-level supportfunction of the apparatus.
 13. The apparatus of claim 12, wherein thelower-level support function includes a system service of the apparatusor a hidden function of the apparatus.
 14. A method of controlling aradar system of a device, the method performed by the device andcomprising: receiving inertial data from an inertial measurement unit ofthe device; and configuring, based on the inertial data and via at leastone or more context-sensitive transition functions, the device to one ofmultiple radar system gating states, the radar system detectingradar-based gestures on behalf of application subscribers, the multiplestates including: a no-gating radar system gating state in which theradar system is enabled to output indications of the radar-basedgestures to the application subscribers; and a soft-gating state inwhich the radar system is prevented from outputting the indications ofthe radar-based gestures to one or more of the application subscribersby shielding a gesture-recognition model from radar data collected bythe radar system.
 15. The method of claim 14, wherein the multiplestates further include a hard-gating state in which the radar system isprevented from detecting the radar-based gestures.
 16. The method ofclaim 14, wherein the soft-gating state prevents the radar system fromoutputting the indications of the radar-based gestures to a first set ofthe application subscribers and a second set of the applicationsubscribers.
 17. The method of claim 16, wherein the first set of theapplication subscribers and the second set of the applicationsubscribers are determined based on one or more device contexts.
 18. Themethod of claim 14, wherein the soft-gating state enables indications ofthe radar-based gestures to be used for a lower-level support functionof the device.
 19. The method of claim 18, wherein the lower-levelsupport function includes a system service of the device or a hiddenfunction of the device.
 20. A non-transitory computer-readable storagemedia storing computer-executable instructions that, when executed by atleast one processor, cause the at least one processor to: receiveinertial data from an inertial measurement unit; and configure, based onthe inertial data and via at least one or more context-sensitivetransition functions, a device to one of multiple radar system gatingstates, a radar system of the device detecting radar-based gestures onbehalf of application subscribers, the multiple states including: ano-gating radar system gating state in which the radar system is enabledto output indications of the radar-based gestures to the applicationsubscribers; and a soft-gating state in which the radar system isprevented from outputting the indications of the radar-based gestures toone or more of the application subscribers by shielding agesture-recognition model from radar data collected by the radar system.